with  an  inti 

WILLIAIV 


Report  Documentation  Page 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 

1 .  REPORT  DATE  2.  REPORT  TYPE 

MAR  1999  N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

Acquisition  for  the  21st  Century:  The  F-22  Development  Program 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

William  J.  /Perry 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

National  Defense  Univesrity  Institute  for  National  Strategic  Studies  Fort 
McNair  Washington,  DC  20319 

8.  PEREORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIEICATION  OE:  17.  LIMITATION  OE 

APl9!TR  apt 

18.  NUMBER  19a.  NAME  OE 

rtln  DAninQ  DTnQPrtNTQTTiT  In  PlnDQ/TM 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  jj-jj 

unclassified  unclassified  unclassified 

204 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Acquisition 
for  the 
21'*  Century: 

The  F-2  2 
Developmen 


Program 


Acquisition 
for  the 

21'*  Century: 

The  F-2  2 

Development 

Program 


with  an  introduction  by 

WILLIAM  J.  PERRY 


NATIONAL  DEFENSE  UNIVERSITY  PRESS 
Washington,  D.C.  1999 


The  Institute  for  National  Strategic  Studies  (INSS)  is  a  major  component  of  the 
National  Defense  University  (NDU),  which  operates  under  the  supervision  of  the 
President  of  NDU.  It  conducts  strategic  studies  for  the  Secretary  of  Defense, 
Chairman  of  the  Joint  Chiefs  of  Staff,  and  unified  commanders  in  chief;  supports 
national  strategic  components  of  NDU  academic  programs;  and  provides  outreach 
to  other  governmental  agencies  and  the  broader  national  security  community. 

The  Publication  Directorate  of  INSS  publishes  books,  monographs,  reports, 
and  occasional  papers  on  national  security  strategy,  defense  policy,  and  national 
military  strategy  through  NDU  Press  that  reflect  the  output  of  NDU  research  and 
academic  programs.  In  addition,  it  produces  the  INSS  Strategic  Assessment  and 
other  work  approved  by  the  President  of  NDU,  as  well  as  Joint  Force  Quarterly,  a 
professional  military  journal  published  for  the  Chairman. 

Opinions,  conclusions,  and  recommendations  expressed  or  implied  with¬ 
in  are  solely  those  of  the  author,  and  do  not  necessarily  represent  the 
views  of  the  National  Defense  University,  the  Department  of  Defense,  or 
any  other  U.S.  Government  agency.  Cleared  for  public  release;  distribu¬ 
tion  unlimited. 

Portions  of  this  book  may  be  quoted  or  reprinted  without  permission,  provided  that 
a  standard  source  credit  line  is  included.  NDU  Press  would  appreciate  a  courtesy 
copy  of  reprints  or  reviews. 


NDU  Press  publications  are  sold  by  the  U.S.  Government  Printing  Office.  For 
ordering  Information,  call  (202)  512-1800  or  write  to  the  Superintendent  of 
Documents,  U.S.  Government  Printing  Office,  Washington,  DC  20402. 


Library  of  Congress  Cataloging-in-Publication  Data 

Acquisition  for  the  21st  Century:  The  F-22  Development  Program 
Michael  D.  Williams 
p.  cm. 

ISBN  1-57906-015-3 

1.  United  States.  Air  Force-Procurement.  2.  F-22  (Jet  fighter  plane)- 
Purchasing.  3.  United  States.  Dept,  of  Defense-Procurement.  4.  Defense  con- 
tracts-United  States.  5.  United  States-Armed  Forces-Weapons  systems- 
Productlon  control.  6.  New  products-United  States.  I.  Defense  Acquisition 
University  (U.S.)  II.  Title.  III.  TitleiAcquisition  for  the  twenty-first  century 
UG1123  .W55  1999 
358.4’16212-dc21 

98-33954 

CIP 

First  Printing,  March  1999 


Dedicated  to  Lieutenant  General  Edward  P.  Barry,  Jr,, 
USAF,  an  outstanding  leader,  commander,  and  acquisition 
professional  who  taught  a  generation  of  Air  Force  officers 
the  meaning  of  duty,  honor,  and  country. 


Contents 


Introduction .  xi 

Foreword .  xiii 

Preface .  xv 

Acknowledgments .  xix 

Part  1.  Theory .  1 

Chapter  1.  Setting  the  Stage .  2 

Chapter  2.  The  20  Principles  of  Acquisition .  8 

Chapter  3.  Risk  Management  and  Concurrent .  22 

Leadership 

Part  IL  Application .  37 

Chapter  4.  The  Formation  of  the  F-22  Team .  38 

Chapter  5.  Defining  and  Refining  the  Requirements  ...  48 

Chapter  6.  Managing  the  Technology .  54 

Chapter  7.  Management  and  Integrated  Product .  76 

Development 

Chapter  8.  Management  and  the  Integrated  Tool  Set ..  92 

Chapter  9.  The  F-22  Program:  How’s  It  Working? .  110 

Part  III.  Beyond  the  F-22 .  129 

Chapter  10.  The  Impact  of  the  F-22  Initiatives .  130 

Chapter  11.  Conclusion: .  142 

Meeting  the  Challenge  of  Complexity 

Appendixes .  155 

A.  A  Brief  History  of  the  Advanced  Tactical  Fighter .  156 

B.  The  F-22  Test  Process .  162 

C.  Prototypes .  166 

D.  Source  Selection:  Concepts  and  Lessons  Learned .  174 


ix 


Introduction 

by 

William  J.  Perry 

The  Department  of  Defense  has  recently  charted  a  new  course  to 
further  improve  the  way  in  which  it  provides  weapon  systems  for  the 
warfighter.  This  path  has  seen  a  fundamental  change  in  how  mem¬ 
bers  of  the  Government/industry  acquisition  team  work  together  to 
develop  Americans  arsenal  for  defense.  These  changes  center  around 
the  use  of  Integrated  Process  and  Product  Development  concepts 
and  the  use  of  empowered,  multifunctional  teams,  called  Integrated 
Product  Teams.  Integrated  Product  Teams  have  been  used  success¬ 
fully  in  industry  for  a  number  of  years.  One  particularly  successful 
example  has  been  the  use  of  Integrated  Product  Teams  on  the 
Boeing  777  program.  Now  several  programs  in  the  Defense  Depart¬ 
ment  are  operating  under  these  concepts.  The  earliest,  and  the 
leader  in  these  concepts,  is  the  Air  Force  F-22  Advanced  Tactical 
Fighter  development  program. 

In  this  book.  Lieutenant  Colonel  Michael  D.  Williams,  USAF,  first 
describes  the  principles  and  the  concepts  of  the  F-22  program.  These 
thoughts  and  ideas  are  broad  in  nature  and  applicable  to  all  weapons 
development  progreuns.  He  then  describes  how  the  F-22  program 
implemented  this  theory.  Here  you  will  find  detailed  explanations  of 
how  the  F-22  program  operates  and  see  practical  examples  of  how  the 
theoretical  concepts  can  be  applied  in  the  real  world.  In  the  book^s 
final  section.  Colonel  Williams  shows  several  examples  of  how  we 
have  successfully  applied  these  concepts  throughout  the  Department. 

I  believe  that  to  truly  improve  the  Defense  Department’s  acquisi¬ 
tion  process  we  must  do  much  more  than  reform  rules  and  regula¬ 
tions.  True  change  will  come  only  from  initiatives  taken  by  the  out¬ 
standing  men  and  women  serving  in  program  offices,  headquarters 
staffs,  development  laboratories,  testing  facilities,  contractor  plants, 
and  throughout  industry.  This  book  explains  successful  program 
management  initiatives  that  will  improve  the  way  the  Department  of 
Defense  provides  for  its  operational  forces.  This  example  can  serve  as 
a  model  and  as  an  inspiration  to  all  of  us. 


William  J.  Perry  served  as  Secretary  of  Defense  from  1994  to  1997. 


Foreword 


The  F-22  development  program  has  served  as  a  flagship  for 
defense  acquisition  reform.  In  this  program,  fundamental  concepts 
such  as  integrated  product  development  using  integrated  product 
teams  were  created,  tested,  refined,  and  fully  implemented.  These 
and  other  ideas  from  the  F-22  program  have  propagated  throughout 
the  Department  of  Defense  and  defense  industry.  Examples  abound 
in  such  programs  as  the  Joint  Strike  Fighter,  the  New  Attack  Subma¬ 
rine,  and  the  Landing  Platform  Dock  (LPD)-17. 

The  F-22  management  approach  has  yielded  much  nearer  term 
benefits  as  well.  Today,  the  Air  Mobility  Command  operates  the  C-17 
airlifter  responding  to  missions  that  span  the  globe.  In  stark  contrast 
to  earlier  predictions,  the  Boeing  Company  is  delivering  this  airlifter 
below  projected  cost  and  ahead  of  schedule — all  as  a  direct  result  of 
much  hard  work  on  the  part  of  government  and  industry  to  implement 
management  reforms  flowing  out  of  the  F-22  program. 

In  Acquisition  for  the  21st  Century:  The  F-22  Development 
Program,  Lieutenant  Colonel  Michael  D.  Williams  captures  the  ele¬ 
ments  which  have  spelled  success  for  this  new  fighter  program. 
However,  this  is  much  more  than  a  documentation  of  history.  Part  I 
describes  the  theory  behind  the  F-22  program  acquisition  innovations. 
Part  II  demonstrates  how  the  program  leadership  applied  this  theory 
to  the  actual  tools  and  techniques  used  to  run  the  fighter  development 
effort.  Part  III  shows  how  the  theory  has  been  applied  to  several 
acquisitions  throughout  the  Department  of  Defense,  proving  that  the 
theory  is  applicable  well  beyond  the  F-22  program. 

Acquisition  for  the  21st  Century  serves  as  both  a  history  and  an 
acquisition  primer  for  successful  program  management.  I  commend  it 
to  Program  Managers  for  their  study.  Perhaps  most  importantly,  I 
commend  it  to  the  men  and  women  in  the  program  offices  and  defense 
suppliers  throughout  the  Nation  and  the  world  who  continue  to  devel¬ 
op  the  world's  finest  equipment  for  the  world’s  best  military. 


Thomas  Crean 
President 

Defense  Acquisition  University 
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Preface 


The  history  of  acquisition  is  characterized  by  dedicated  individu¬ 
als  working  within,  and  sometimes  around,  the  system  to  provide 
weapons  to  the  warfighter  in  the  best  way  possible.  Few  outside  our 
often-maligned  defense  procurement  process  know  of  the  many 
success  stories  in  the  weapons  developed  by  the  United  States. 
Admittedly,  a  few  highly  visible  programs  have  fallen  far  short  of 
being  considered  successful.  The  hallmark  of  every  development  effort 
is  the  desire  of  the  program’s  management  to  learn  from  those  who 
went  before,  capitalize  on  their  lessons  learned,  and  implement 
new  procedures  to  improve  the  process  of  developing  a  system  that 
meets  the  warfighter’s  needs.  The  development  of  America’s  next- 
generation  air-superiority  fighter — the  Air  Force  F-22 — ^is  an  example 
of  a  program  that  has  done  just  that:  capitalized  on  past  successes, 
heeded  warnings  from  past  failures,  created  new  tools  and  methods, 
and,  as  a  result,  developed  a  model  acquisition  program. 

This  account  describes  new  methods  and  processes  used  in  devel¬ 
oping  the  F-22.  What  are  these  innovations,  and  why  would  readers 
want  to  know  about  them? 

In  1986,  General  Larry  D.  Welch,  the  Air  Force  Chief  of  Staff, 
singled  out  the  F-22 — then  called  the  Advanced  Tactical  Fighter 
(ATF) — program  as  an  opportunity  to  improve  the  acquisition  process. 
This  book  explains  how  leaders  of  the  program  have  met  that  man¬ 
date;  what  they’ve  done  to  improve  the  process  of  designing  and  devel¬ 
oping  a  weapon  system.  But,  again,  why  is  it  important  for  readers  to 
know  about  the  F-22  program? 

By  picking  up  this  study  you  are  obviously  interested  in  acquisi¬ 
tion.  The  F-22  system  program  office  hosts  more  than  100  visitors 
every  year;  they,  like  you,  share  an  interest  in  acquisition.  The  visi¬ 
tors  have  included  program  managers  from  within  the  Air  Force, 
from  other  services,  and  from  elsewhere  within  the  Department  of 
Defense  (DOD)  as  well  as  congressional  staffers,  procurement  offi¬ 
cials  from  agencies  outside  the  DOD  (such  as  the  National 
Aeronautics  and  Space  Administration  and  the  Federal  Aviation 
Administration),  and  officials  from  other  countries.  All  have  wanted 
to  know  how  the  F-22  program  operates. 

Perhaps  you’re  an  acquisition  official  trying  to  see  what  all  the 
hoopla  is  about.  Perhaps  you’re  trying  to  reform  the  acquisition 
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process.  Maybe  youVe  a  program  manager  merely  trying  to  survive 
to  see  another  sunrise.  Whatever  your  position,  this  book  tells  how 
the  F-22  program  operates.  More  important,  it  shows  why  the  F-22 
program  runs  the  way  it  does.  Before  diving  in,  however,  you  need  to 
know  the  answers  to  several  questions. 

What  Are  the  Book’s  Conventions? 

The  subject  of  this  study  has  known  two  names — the  ATF 
program  and  the  F-22  program.  In  October  1986,  to  pursue  a 
replacement  for  the  F-15  fighter,  the  Air  Force  awarded  two  air  vehi¬ 
cle  design  contracts  for  the  demonstration/validation  phase.  This 
effort  pitted  Lockheed  Aircraft  Systems  Company  (teamed  with  the 
General  Dynamics  Corporation  and  Boeing  Aircraft  Company) 
against  Northrop  Aircraft  Division  (teamed  with  McDonnell  Douglas 
Corporation)  to  conduct  design  and  technology  demonstrations  in 
order  to  validate  their  weapon  system  approaches  for  the  next- 
generation  air-superiority  fighter.  These  approaches  stemmed  from 
the  previous  concept  development  investigation  phase,  Lockheed’s 
protot5q)e,  designated  the  YF-22A,  and  the  YF-23A  Northrop,  were  to 
fly  and  demonstrate  their  respective  team’s  predicted  perform¬ 
ance.  Simultaneously,  the  ATF  program  office  took  control  of 
the  ATF  Engine  program,  previously  called  the  Joint  Advanced 
Fighter  Engine  program.  This  engine  program  similarly  followed  a 
competitive  demonstration/validation  prototype  program  in  which 
General  Electric  Aircraft  Engine  Company  vied  with  Pratt  & 
Whitney  Government  Engine  Business  to  see  who  would  develop 
and  produce  the  ATF’s  engine. 

In  April  1991,  Secretary  of  the  Air  Force  Donald  B.  Rice  selected 
the  Lockheed  F-22  team  to  develop  the  weapon  system  (the  fighter 
plus  the  training  and  support  systems).  He  also  selected  Pratt  & 
Whitney  to  design  £ind  build  the  ATF’s  F119  engine.  (See  appendix  A 
for  a  brief  history  of  the  program.) 

Following  Secretary  Rice’s  decision,  the  engineering  and  manu¬ 
facturing  development  phase  started  on  August  2,  1991.  At  that  time 
the  ATF  program  became  the  F-22  program.  This  account  refers  to  the 
ATF  program  whep  discussing  the  early  program  (such  as  the  demon¬ 
stration/validation  phase)  and  the  F-22  program  when  describing 
engineering  and  manufacturing  development,  the  subsequent  phase. 
Also,  several  of  the  main  participants  changed  rank  during  the  course 
of  events.  For  example,  James  A.  Fain,  Jr.,  the  Program  Director  from 
1986  until  1992,  started  the  program  as  a  colonel  and  retired  from  the 
Air  Force  in  1995  as  a  lieutenant  general.  I  refer  to  him  as  Greneral 
Fain  throughout  the  text. 
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Who  Originated  the  Ideas  in  the  Book? 

The  ideas,  thoughts,  and  collective  wisdom  presented  come  from 
the  founders  of  the  ATP  program:  General  Fain,  Eric  E.  Abell,  Colonel 
Wallace  T.  Bucher,  and  Jon  T.  Graves,  whose  positions  in  the  program 
appear  below. 

These  four  leaders  drew  support  from  an  able  staff  of  some  of  the 
Air  Force’s  and  industry's  frnest  personnel.  The  entire  ATF/F-22  team 
had  a  hand  in  creating  and  shaping  the  concepts  described.  That  team 
consists  of  members  of  the  program  office,  representatives  of  the  opera¬ 
tional  user  (Air  Combat  Command,  formerly  Tactical  Air  Command), 
representatives  of  other  Government  organizations,  and,  significantly, 
members  of  the  contractor  community  (the  F-22  and  F-23  aircraft  teams 
and  the  F119  and  F120  engine  teams).  Among  this  team,  Colonel  Robert 
Kayuha,  Colonel  William  Buzzell,  Michael  Nock,  Ronald  Runkle,  and 
Lieutenant  Colonel  Janet  Bloom  stand  out  as  major  contributors  to  the 
acquisition  innovations  now  in  use  by  the  program. 

I - L 


1986-1992 

1985- 1993 

1986- 1991 
1984-1996 


James  A.  Fain,  Jr. 
Eric  E.  Abell 
Wallace  T.  Bucher 
Jon  T.  Graves 


The  ATF  Program  Founders 

Program  Director 
Technical  Director 
Deputy  Program  Director 
Deputy  Program  Director 


What  Is  My  Perspective? 

I  joined  the  ATF  program  as  a  project  officer  in  January  1990,  at 
the  end  of  the  demonstration/validation  phase.  In  March  1991,  I 
became  General  Fain’s  executive  officer,  gaining  firsthand  access  to 
the  program  director  and  his  staff.  I  left  the  program  for  14  months 
but  returned  in  September  1993  to  head  the  Support  System 
Integrated  Product  Team,  one  of  the  F-22’s  four  major  teams.  In  that 
position,  I  had  to  live  or  die  with  the  tools  and  procedures  described 
in  chapters  7  and  8. 

Thus,  during  my  two  tours  with  the  F-22  program,  I  saw  the 
program  from  the  perspective  of  the  program  director  and  his  key 
advisors  and  then  participated  in  day-to-day  program  execution. 
Admittedly  a  biased  observer,  IVe  seen  firsthand  how  the  program 
operates.  Moreover,  I’ve  worked  with  the  team  that  laid  the  founda¬ 
tion  for  the  program. 
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How  Is  THE  Book  Laid  Out? 

Simply  put,  it’s  theory  first,  practice  second,  and  other  examples 
third.  Don’t  bypass  the  theory,  or  what  I  call  the  principles  and 
methods,  Here’s  why:  The  way  the  F-22  program  carries  out  its 
mission  works  well  for  the  F-22.  However,  its  solutions  are  system 
specific.  They  meet  the  unique  needs  of  the  F-22  program — or  a 
similar  program  of  its  size  and  in  the  same  environment.  None¬ 
theless,  the  F-22’s  fundamental  concepts  should  apply  to  most  acqui¬ 
sitions  in  most  environments.  Thus,  the  section  on  theory  (part  I) 
gives  basic  acquisition  truths — the  really  important  material.  Their 
application  (part  II)  shows  how  the  F-22  program  operates  and  gives 
examples  that  help  prove  the  value  of  the  underlying  theory.  Part  III 
describes  several  programs  and  activities  within  the  DOD  that  are 
using  these  techniques  today  and  the  way  the  Department  has  man¬ 
dated  their  use  in  all  aspects  of  weapon  acquisition  for  the  future. 
For  those  who  are  especially  hearty,  more  detailed  discussion  on 
some  special  topics  follows  in  the  appendixes. 

How  Best  to  Read  This  Book? 

I  suggest  that  you  read  the  conclusion  (chapter  11)  first.  It  sum¬ 
marizes  the  entire  book  and  includes  the  most  important  points.  Then 
read  the  preface  and  the  first  page  of  each  chapter.  At  the  beginning 
of  each  chapter  is  an  outline  of  that  chapter,  which  you  can  scan  to 
find  the  information  you  most  want  to  read  in  more  depth.  Chapters  2 
and  3,  on  principles  and  methods,  present  the  heart  of  the  basic  acqui¬ 
sition  truths,  and  chapter  9  describes  the  lessons  learned  from  the  use 
of  these  ideas  on  the  F-22  program. 
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Photo  courtesy  of  Lockheed-Martin. 


The  YF-22  (foreground)  and  the  YF-23  advanced  tactical  fighter  prototypes  in  formation. 
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Theory 

Because  circumstances  vary,  there’s  some  danger  in  assuming 
that  the  same  procedures  that  have  been  successful  in  one 
weapon  development  program  are  appropriate  for  another.  If 
the  environment  and  all  conditions  are  the  same  in  both  programs, 
such  an  approach  may  work.  However,  when  the  situations  differ 
(as  is  usual),  the  unchallenged  application  of  original  prescriptions 
will  likely  fail.  Part  I  examines  basic  acquisition  truths  that  are 
appropriate  for  most  conditions  and  environments.  Twenty  princi¬ 
ples  make  up  the  first  part  of  these  truths  (described  in  chapter  2). 
Two  methods  complement  them  (chapter  3).  The  Advanced  Tactical 
Fighter  (ATF)  program’s  leadership  used  these  concepts  to  conduct 
the  ATF  demonstration/validation  phase  and  to  establish  the  F-22 
engineering  and  manufacturing  development  phase. 


1 


1 


Setting  the  Stage 


•  Background:  The  -Ugl’t  Yeaiw  OF  Weafon 
DEVELOPMimr 

•  Congressional  AND  DODBefchim 

•  The  Charge:  Create  a  Model  Acquisition. 
Programl 


2 


Congress  has  warmly  embraced  well-disciplined,  trim  and  fit, 
ready  to  fight  programs  such  as  the  advanced  tactical  fighter 
(ATF),  now  known  as  the  F-22, 


Senator  John  W.  Warner, 
“Legislative  Perspective,'' 
in  Military  Project  Management  Handbook 


Congress,  the  Department  of  Defense  (DOD)  leadership  and, 
most  important,  the  Air  Force  warfighters  view  the  F-22  devel¬ 
opment  effort  as  a  model  acquisition  program.  It  is  instructive 
to  remember  that  the  first  two  groups  have,  in  the  past,  frequently 
seen  much  to  fix  within  the  weapon  development  process.  What  have 
been  the  problems,  and  what  role  has  the  F-22  program  played  in 
helping  improve  the  way  America  builds  weapons? 

Background:  The  Ugly  Years  of  Weapon  Development 

Interest  in  improving  the  acquisition  process  for  U.S.  military 
weapons  and  supplies  began  with  the  founding  of  the  Nation.  However, 
since  World  War  II,  the  executive  and  legislative  branches  have  shown 
an  exponentially  increasing  interest  in  military  acquisitions. 

Fear  of  the  Soviet  threat  in  the  1950s  spurred  U.S.  development 
of  weapons  that  would  provide  the  Armed  Forces  a  military  edge. 
DOD  acquired  weapon  systems  with  a  strong  focus  on  perform¬ 
ance,  at  the  expense  of  cost  and  schedule.  In  1962,  the  Harvard 
Weapons  Acquisition  Research  Project  examined  12  major  defense 
projects  and  found  a  tendency  for  cost  and  schedule  overruns 
(costs  averaged  seven  times  more  than  initially  estimated  and 
development  took  36%  longer  than  originally  scheduled).  The 
Project  recommended: 

•  Making  cost  equal  in  importance  to  performance  and  schedule 
•  Eliminating  “gold  plating"  (developing  performance  and  featmes  well 
beyond  that  reasonably  needed  for  effective  military  capability) 

•  Increasing  competition  at  the  start  of  a  new  project 
•  Reducing  the  number  of  cost-type  contracts,  particularly  cost- 
plus-fixed-fee  contracts. 
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By  the  mid-1960s,  in  an  effort  to  improve  the  acquisition  process, 
Robert  H.  Charles,  the  Assistant  Secretary  of  Defense  for  Installations 
and  Logistics,  conceived  of  the  total  package  procurement  concept.  His 
motivation  was  to 

•  Reduce  cost  ovemms 

•  Instill  greater  competition 

•  Assign  the  contractor  total  responsibility  for  system  design 

•  Prevent  contractors  from  ‘iDuying  into”  programs  by  submitting 
artificially  low  bids. 

Although  the  total  package  procurement  concept  had  some 
limited  success  (the  Air  Force  Maverick  air-to-surface  missile  is 
the  best  example),  it  did  not  solve  the  traditional  problem  of  cost 
overruns.  In  addition,  programs  continued  to  slip  schedules  and 
frequently  did  not  meet  their  performance  specifications.  Programs 
that  encountered  problems  under  the  concept  included  the  Air  Force 
C-5A  cargo  aircraft,  the  Army  AH-56A  Cheyenne  helicopter,  and  the 
Navy  DD-963  destroyer. 

Congressional  and  DOD  Reform 

The  problems  on  the  C-5A,  AH-56A,  and  DD-963  programs  raised 
concern  in  Washington  about  the  federal  acquisition  process  and  led 
to  numerous  studies,  including  that  of  the  Commission  on 
Government  Procurement,  established  by  Congress  in  November 
1969.  Also  in  reaction  to  the  problems  of  cost,  schedule,  and  technical 
performance,  the  Deputy  Secretary  of  Defense,  David  Packard  (the 
co-fo\mder  of  the  successful  electronics  giant  Hewlett-Packard),  initi¬ 
ated  a  host  of  efforts  to  reform  the  Department’s  method  of  developing 
and  acquiring  weapons.  These  included: 

•  Establishing  a  Defense  Systems  Acquisition  Review  Coimcil  (the 
predecessor  of  the  current  Defense  Acquisition  Board)  to  examine 
and  assess  a  program’s  readiness  to  proceed  to  the  next  phase  of 
its  development 

•  Increasing  competition  through  the  use  of  prototype  systems  to 
decrease  reliance  on  technically  imreliable  “paper”  competitions 

•  Encouraging  early  testing  and  evaluation  of  a  system  to  gauge  its 
performance  and  limitations  and  thus  its  operational  suitability 
as  soon  as  possible 

•  Requiring  formal  Secretarial  approval  at  three  points  in  the 
weapon  development  cycle  known  as  milestones  (between  the  four 
phases  of  the  cycle:  concept  formulation,  validation,  operational 
systems  development,  and  production;  see  Figure  1-1). 
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Figure  1-1.  Original  Packard  Weapon  Development  Phases,  1970 
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In  1972,  following  Packard’s  changes  to  DOD  procurement,  the 
Commission  on  Government  Procurement  released  its  four-volume 
report,  which  made  149  recommendations — 82  requiring  executive 
action  and  67  requiring  legislative  action.  Among  the  major  recom¬ 
mendations  were: 

•  Emphasize  up-front  competition  on  alternative  system  approaches 

•  Simplify  the  decision  process  by  rel5dng  more  on  soimd  judgment 
and  less  on  regulations  and  complicated  contracts 

•  Reduce  the  layers  of  management  and  administration  between 
policymakers  and  program  offices 

•  Require  more  government  rehance  on  the  private  sector  and  less 
on  in-house  facilities  for  procurement 

•  Give  visibility  to  Congress  so  that  it  can  exercise  its  responsibili¬ 
ties  (i.e.,  provide  Congress  with  the  information  it  needs  to  make 
key  program  decisions  and  commitments). 

DOD  had  already  implemented  many  of  the  recommendations  by 
the  time  the  Commission  released  the  report.  One  major  change  lay  in 
the  search  for  alternate  ways  to  eliminate  mission  deficiencies.  This 
resulted  in  a  new  milestone  (termed  Milestone  0,  so  that  the  existing 
milestones  would  not  be  renumbered)  that  required  the  approval  of  a 
need  statement  before  the  start  of  concept  exploration. 

The  1970s  saw  continued  acquisition  reviews  and  studies.  In 
1976,  the  Office  of  Management  and  Budget  issued  Circular  A- 109, 
which  defined  the  federal  acquisition  process  and  aligned  the  majori¬ 
ty  of  executive  departments  with  the  DOD  phased  procurement 
method  (including  Milestone  0). 

In  1981,  Frank  C.  Carlucci,  the  Deputy  Secretary  of  Defense, 
directed  changes  to  the  acquisition  system  through  his  31  Initia¬ 
tives.  His  objective  was  to  improve  the  acquisition  system.  His 
goals  were  to: 
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•  Reduce  cost 

•  Improve  the  efficiency  of  the  acquisition  process 

•  Increase  the  stability  of  weapon  development  programs 

•  Decrease  the  time  it  took  to  acquire  military  hardware. 

In  1983,  the  spotlight  again  fell  on  federal  and,  specifically,  DOD 
procurement.  Kenneth  L.  Adelman  and  Norman  R.  Augustine,  in  The 
Defense  Revolution,  describe  the  times  this  way: 

Stories  of  wildly  inflated  prices  for  ordinary  items  [toilet 
seats,  coffee  pots,  diodes,  pliers,  and  hammers]  first  sur¬ 
faced  in  the  early  years  of  the  Reagan  defense  buildup  when 
the  then  secretary  of  defense  extensively  publicized  such 
incidents  in  an  apparent  effort  to  demonstrate  that  the 
Defense  Department  would  be  a  scrupulous  and  tough 
manager  of  the  increased  funding  it  was  seeking.  Somehow 
the  whole  effort  backfired.  The  public  concluded  that  the 
department  must  be  grossly  incompetent  to  have  paid  so 
much  money  for  everyday  items  and  that  the  defense  indus¬ 
try  must  be  composed  of  crooks. 

With  the  tortured  history  of  weapon  acquisition  and  the  image  of 
the  infamous  $400  hammer  firmly  in  mind.  President  Ronald 
Reagan  established  his  Blue  Ribbon  Commission  on  Defense 
Management.  This  Commission  brought  David  Packard  back  to  the 
center  stage  of  DOD  acquisition  reform.  The  Acquisition  Task  Force 
(headed  by  William  J.  Perry),  a  subgroup  of  the  Commission, 
released  its  initial  findings  and  recommendations  in  A  Formula  for 
Action:  A  Report  to  the  President  on  Defense  Acquisition  on  April  7, 
1986.  The  study  found  that  cost  and  schedule  overruns  still  existed. 
The  task  force  summarized  the  issues  as  follows:  “All  too  many  of 
our  weapon  systems  cost  too  much,  take  too  long  to  develop,  and,  by 
the  time  they  are  fielded,  incorporate  obsolete  technology.”  The  key 
recommendations  of  the  study  included: 

•  Streamline  the  organization  and  procedures  of  acquisition 

•  Use  technology  to  reduce  the  cost  of  acquisition  (by  stressing 
the  importance  of  prototypes  and  demonstrations  that  prove  the 
effectiveness  of  technology) 

•  Balance  cost  and  performance  (by  encouraging  early  trade-offs  to 
meet  military  needs  at  a  lower  cost) 

•  Stabilize  acquisition  programs 

•  Expand  the  use  of  commercial  products 

•  Increase  the  use  of  competition  in  the  acquisition  process 

•  Enhance  the  quality  of  acquisition  personnel 

•  Improve  the  nation’s  capability  for  industrial  mobilization. 
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The  Charge:  Create  a  Model  Acquisition  Program 

The  $400  hammer  and  the  Packard  Commission’s  report  set  the 
tone  in  the  halls  of  the  Pentagon  in  December  1986.  The  newly 
appointed  director  of  the  Air  Force  program  to  develop  the  next- 
generation  air-superiority  fighter,  General  James  A.  Fain,  Jr.  (then  a 
colonel),  was  summoned  to  meet  with  General  Larry  D.  Welch,  the  Air 
Force  Chief  of  Staff.  General  Welch  was  quick  and  to  the  point  in  his 
charge  to  Fain: 

1.  Develop  the  Air  Force  replacement  for  the  aging  F-15  fighter 

2.  Create  a  model  acquisition  program  and  thereby  set  the 
standard  for  all  future  weapon  developments. 

General  Fain  had  just  served  as  the  Director  of  the  Low-Altitude 
Navigation  and  Targeting  Infrared  for  Night  (LANTIRN)  program  (he 
is  credited  with  bringing  it  back  from  the  brink  of  disaster).  As  such, 
he  understood  the  first  task,  to  lead  the  program  that  would  develop 
and  field  an  F-15  replacement.  However,  his  second  task,  to  create  a 
model  acquisition  program,  seemed  daunting,  perhaps  even  impossi¬ 
ble.  As  he  flew  out  of  Washington  on  his  way  back  to  Wright-Patterson 
Air  Force  Base,  Ohio,  he  decided  he  would  tackle  the  comprehensible 
issue  first.  He’d  figure  out  the  model  acquisition  part  later. 
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The  20  Principles 
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•  Tbe  Principles  OF  AcQWsntioN  ,  : 

•  Summary 
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It's  not  what  we  did  that's  important  It's  why  we  did  what  we 
did  that  people  should  remember! 


General  James  A.  Fain,  Jr. 


When  General  Fain  became  the  Director  of  the  Advanced 
Tactical  Fighter  (ATF)  development  program,  he  brought 
with  him  a  list  of  what  he  would  avoid  in  managing  an 
acquisition  program.  He  had  created  the  list  from  hard- won  experi¬ 
ence  running  the  Low  Altitude  Navigation  and  Targeting  Infrared 
for  Night  (LANTIRN)  program.  For  example,  on  LANTIRN,  he  had 
become  frustrated  by  its  failure  to  integrate  the  cost-reporting 
system  with  the  statement  of  work.  Unable  to  track  actual  cost 
performance  by  task,  he  found  the  cost-reporting  tool  of  little  value. 
He  vowed  then  never  to  run  another  program  without  properly  inte¬ 
grating  its  various  management  tools. 

Fain,  and  the  principal  members  of  his  team,  Eric  E.  Abel, 
Colonel  Wallace  T.  Bucher,  and  Jon  T.  Graves,  examined  the 
successes  of  earlier  weapon  development  programs,  such  as  those 
for  the  F-15  and  the  F-16  fighters.  They  also  looked  at  where  these 
and  other  programs  had  run  into  trouble.  Armed  with  the  LANTIRN 
list  and  lessons  learned  from  other  programs,  they  drew  on  their 
accumulated  acquisition  experience  to  evolve  a  set  of  operating 
principles  for  managing  the  ATF/F-22  program.  Using  these  operat¬ 
ing  principles  as  the  foundation  for  their  decisions  and  actions, 
they  hoped  to  create  a  program  that  would  successfully  yield  an 
air-superiority  fighter  responding  to  the  needs  of  the  Tactical  Air 
Command-  Figure  2-1  lists  the  operating  principles  they  defined. 

The  Principles  of  Acquisition 

1.  Operate  with  Integrity  —  Integrity  stands  out  as  the 
conspicously  most  important  principle.  Webster's  New  World 
Dictionary  defines  integrity  as  “the  quality  or  state  of  being  of  sound 
moral  principle;  uprightness,  honesty,  and  sincerity.”  Defined  for  the 
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Figure  2-1.  The  ATF/F-22  Program’s  Principles  of  Acquisition 

1.  Operate  with  integrity. 

2.  Work  as  a  team. 

3.  Use  logic  and  common  sense. 

4.  involve  everyone. 

5.  Integrate  the  entire  system. 

6.  Ensure  ownership. 

7.  Use  a  disciplined  approach. 

8.  Understand  what  is  really  required. 

9.  Set  realistic  expectations  and  meet  them. 

10.  Provide  realistic  options. 

11.  Take  a  long-term  view. 

12.  Do  it  right  the  first  time. 

13.  Have  what  you  need  for  the  effort. 

14.  Ensure  everyone  knows  what  it  takes  to  meet  the  goal. 

15.  Use  an  event-based  schedule  with  defined  success  criteria. 

16.  Define  success  and  be  able  to  measure  it. 

17.  Reward  success. 

18.  Focus  on  a  win-win  relationship, 

19.  Guarantee  open  communications. 

20.  Achieve  success  with  a  positive  attitude  and  focus. 


F-22  program,  integrity  went  farther.  It  demanded  two  things  of  every 
team  member: 

•  Conduct  all  business  relationships  honestly 

•  Make  sure  the  receiver  understands  one’s  message. 

General  Fain  amplified  the  second  point.  On  more  than  one  occa¬ 
sion,  he  had  seen  those  who  would  intentionally  present  factual 
details  in  such  a  way  that  the  person  receiving  the  information 
would  incorrectly  assume  a  certain  condition  to  be  true.  Although 
knowing  that  the  information  could  (and  likely  would)  be  misinter¬ 
preted,  the  presenters  technically  would  not  be  lying.  Fain  required 
all  members  of  the  ATF/F-22  team  to  not  only  present — but  also 
make  sure  that  the  receiver  understand — the  full,  true  story. 

2.  Work  as  a  Team  —  Teamwork  seems  a  somewhat  obvious 
notion.  Working  together  and  sharing  ideas  and  suggestions  allow 
many  brains  to  focus  on  a  problem.  Teamwork  also  draws  individuals 
together,  helps  them  bond,  and  helps  commit  them  to  the  endeavor. 


Prinicpies  of  Acquisition  1 1 


The  program  leadership  recognized  the  enormous  complexity,  techni¬ 
cal  as  well  as  managerial,  of  developing  a  supersonic,  highly  reliable 
stealth  fighter.  They  realized  that  to  succeed  at  this  undertaking  they 
had  to  constantly  reinforce  the  idea  of  teamwork. 

In  designing  the  ATF,  the  attributes  of  stealth,  aerodynamic 
performance,  supercruise  (the  ability  to  fly  faster  than  the  speed  of 
sound  without  using  afterburner),  integrated  avionics,  cost,  reliabili¬ 
ty,  and  ease  of  maintenance  could  all  work  against  one  another.  For 
example,  a  very  low  observable  aircraft  could  be  made  by  ridding  the 
aircraft  of  most  of  the  maintenance  access  doors,  thus  eliminating 
their  cracks  and  gaps  (which  can  reflect  radar  energy  and  make  the 
aircraft  easier  to  detect).  But  that  would  make  the  aircraft  much  more 
difficult  to  service,  because  the  maintenance  technician,  with  fewer 
access  panels,  would  have  a  harder  time  checking  fluid  levels  or  get¬ 
ting  to  parts  that  need  replacement.  The  goal,  then,  is  to  balance  these 
attributes  while  designing  a  fighter  that  meets  all  requirements. 

The  way  to  reach  a  balanced  design  is  to  make  sure  that  the 
various  disciplines  responsible  for  the  attributes  (such  as  reliability 
engineering,  signatxire  engineering,  and  manufacturing  as  well  as 
those  concerned  with  logistics,  finance,  and  contracting)  cooperate, 
that  is,  work  as  a  team.  Each  discipline  needs  to  understand  the  con¬ 
straints  imposed  by  other  disciplines,  and  those  responsible  must 
coordinate  and  interweave  their  work  to  achieve  a  balanced  design. 

The  ATF  management  te£un  realized  the  need  for  teamwork  from 
the  beginning.  So  that  everyone  understood  the  requirements,  the 
team  saw  that  the  government  program  office  and  the  contractors 
worked  closely  with  the  operational  users  (the  combat  fighter  pilots 
who  create  the  requirements).  At  the  same  time,  the  team  leadership 
made  sure  that  the  users  saw  firsthand  what  their  requirements 
meant  in  terms  of  aircraft  complexity  and  cost.  Thus,  teamwork 
applied  to  all  levels  and  members  of  the  team:  the  users,  the  contrac¬ 
tors,  and  the  program  office. 

3.  Use  Logic  and  Common  Sense  —  General  Fain  had  a 
deep-seated  belief  that  the  only  reason  for  rules  and  regulations  was 
to  try  to  prevent  people  from  repeating  past  mistakes.  His  view  was 
that  reasonable  people,  who  were  aware  of  the  intended  direction  of 
the  organization,  would  use  their  common  sense  and  make  the  right 
decisions.  In  other  words,  he  assumed  they  would  make  decisions 
logically,  based  on  the  facts  and  not  on  emotions.  In  his  own  words, 
“If  something  doesn’t  feel  right,  look  right,  smell  right,  it  probably 
isn’t  right.  Don’t  do  it!  Just  use  plain  old  everyday  common  sense  in 
everything  you  do,  and  you’ll  do  the  right  thing.”  He  also  encouraged 
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each  team  member  to  take  on  “the  system”  to  eliminate  rules  or 
regulations  that  failed  the  logic  test. 

4.  Involve  Everyone  —  As  Principle  2  states,  teamwork  is 
important.  But  who  makes  up  the  team?  Simply  put,  anyone  who 
has  a  stake  in  the  outcome  belongs  on  the  team.  For  the  ATF 
program,  the  core,  the  everyday  functioning  team,  consisted  of 
representatives  of  the  contractors,  the  program  office,  and  the 
user — those  working  most  closely  with  the  program  and  thus  having 
a  greater  stake  in  the  outcome. 

However,  the  program  team  also  included  those  outside  this 
immediate  family.  The  program's  leadership  saw  a  need  for  each 
person  who  would  come  in  contact  with  the  ATF  to  be  tied  into  the 
program  so  that  everyone  would  understand  the  nuances  of  the 
program  better.  More  importantly,  the  leadership  wanted  all  team 
members  to  feel  they  had  something  to  gain  by  the  success  of  the  ATF. 
In  earlier  programs,  members  of  various  staff  groups  and  oversight 
agencies  (most  of  the  organizations  outside  the  program  office)  creat¬ 
ed  great  turmoil  through  their  questions  and  requests  to  review  the 
program  because  they  weren’t  intimately  familiar  with  the  programs. 
Many  of  their  perceptions  and  beliefs  were  inaccurate.  These  groups 
did  not  necessarily  share  the  desire  to  see  the  program  succeed 
that  the  user,  the  program  office,  and  the  contractors  did.  The  ATF 
program  management  knew  that  these  groups  formed  a  necessary 
part  of  fielding  a  weapon  system  successfully  and  focused  on  making 
them  an  active,  contributing  part  of  the  ATF  development  team. 

The  program  office  achieved  this  involvement  in  many  ways.  For 
example,  before  release  of  the  final  request  for  proposal  (RFP) — the 
document  that  tells  the  competing  contractors  what  the  Government 
wants  them  to  offer  to  do — the  program  office  called  together  18  Air 
Force  generals  and  Navy  admirals  involved  in  the  ATF  program.  At 
this  General  Officer  Review,  the  program  office  explained  precisely 
what  it  was  requesting  of  the  contractors  and  consulted  the  flag  offi¬ 
cers  for  their  advice  and  insights.  Dissent  by  any  member  of  this 
forum  could  have  easily  delayed  or  complicated  the  procurement. 
However,  the  program  office  had  thoroughly  prepared  the  RFP  and 
the  overarching  acquisition  strategy.  Both  proved  to  be  examples  of 
excellent  coordination  with,  and  involvement  by,  affected  government 
agencies  and  competing  contractor  teams.  For  these  reasons,  there 
was  no  dissent,  and  each  attendee  agreed  to  the  release  of  the  RFP 
without  major  change.  Moreover,  since  the  review  directly  involved 
these  senior  officers  they  not  only  became  part  of  the  process  but  also 
readily  bought  into  the  effort. 
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6.  Integrate  the  Entire  System  —  In  the  LANTIRN  program, 
Greneral  Fain  had  been  concerned  that  the  many  management  tools 
available  (such  as  cost  performance  reports,  work  breakdown  struc- 
tiu’es,  and  specifications)  rarely  related  to  one  another.  These  tools 
frequently  reported  conflicting  information  and,  as  a  result,  often 
worked  at  cross-purposes.  In  addition,  some  information  developed 
and  used  by  the  contractor  personnel,  such  as  internal  cost-account¬ 
ing  information,  did  not  follow  the  regulation-required  government 
format.  As  a  result,  the  contractor  had  to  translate  this  information 
into  that  format.  General  Fain’s  intent  was  to  coordinate  and  focus  adl 
of  the  tools  in  the  acquisition  “tool  set”  to  deliver  a  high-performance, 
high-quality  product  on  schedule  and  within  cost.  Ideally,  the  contrac¬ 
tor  and  the  Government  would  use  the  same  information  (thus  elimi¬ 
nating  the  need  for  translation  of  data). 

As  the  ATF  demonstration/validation  phase  progressed,  the  prin¬ 
ciple  of  integration  expanded.  It  was  a  driving  force  behind  the 
creation  of  the  acquisition  strategy  for  the  next  phase,  engineering 
and  manufacturing  development  (EMD).  The  founding  fathers 
ensured  that  all  the  elements  of  the  acquisition  strategy  were  tied 
together  to  support  the  overall  goal  of  fielding  the  ATF.  They 
described  the  concept  as  a  stone  wall.  Just  as  fine  New  England  stone 
walls  are  carefully  crafted  by  interlocking  each  imiquely  shaped 
stone  to  create  great  strength,  all  the  elements  of  the  ATF  acquisition 
strategy  had  been  carefully  fitted.  If  one  element  (stone)  of  the  strat¬ 
egy  was  taken  out  or  changed,  it  would  affect  the  entire  strategy 
(wall).  Change  was  allowed,  but  the  integrated  process  had  to  recog¬ 
nize  all  the  implications  of  any  change.  The  F-22  integrated  acquisi¬ 
tion  strategy  “stone  wall”  is  shown  in  Figure  2-2. 

6.  Ensure  Ownership  — As  stated  in  the  fourth  principle,  the 
program  leadership  wanted  everyone  involved  to  be  committed  to 
success  of  the  F-22.  Their  logic  was  that,  if  all  team  members  owned 
the  program,  all  would  want  to  see  it  succeed  and  would  do  what  it 
took  to  develop  a  high-quality  product. 

The  F-22  program  office  established  ownership  in  many  ways.  At 
the  unit  level,  the  program  office  structured  the  government  and 
contractor  organizations  around  teams,  called  integrated  product 
teams  (see  chapter  7),  that  focused  on  developing  a  specific  product 
(for  example,  the  cockpit  team  or  the  landing  gear  team).  To  drive 
ownership  down  to  the  individual  level,  the  program  leadership 
gave  the  team  members  ultimate  responsibility  for  their  product  by 
allocating  a  budget,  requirements,  and  a  schedule — making  them 
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Figure  2-2.  The  F-22  Integrated  Acquisition  Strategy 
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“mini-program  directors”  for  their  product.  The  leaders  wanted  all 
team  members  to  know  their  product  so  well  that  they  could  describe 
its  current  status  at  a  moment's  notice  (i.e.,  know  the  technical 
performance  achieved  to  date,  as  well  as  the  latest  production  cost 
and  weight  estimates.) 

The  management  team  also  wanted  to  establish  ownership  among 
participants  who  worked  at  the  various  headquarters  and  associated 
organizations  around  the  coimtry  primarily  by  involving  them  in  the 
full  program.  As  in  the  General  Officer  Review,  members  outside  the 
program  office,  not  traditionally  viewed  as  members  of  acquisition 
teams,  had  an  opportunity  to  become  involved  by  casting  their  vote. 
The  direct  result  was  buy-in  and  greater  ownership  of  the  F-22. 
Pentagon  staff  officers  were  surprised  and  pleased  that  they  were 
invited  to  attend  F-22  reviews.  Members  of  the  Defense  Plant 
Representative  Offices  were  stunned  to  hear  the  Program  Director  say 
they  were  to  be  treated  like  members  of  the  program  office  and  that 
they  were  full  members  of  the  F-22  development  team.  Once  they 
were  convinced  of  this,  ownership  and  its  inherent  desire  for  program 
success  followed  naturally. 

7.  Use  a  Disciplined  Approach  —  Perhaps  a  disciplined 
approach  to  a  program  seems  like  common  sense  (and  it  is),  but  the 
founders  had  seen  many  earlier  programs  fail  for  lack  of  a  structured 
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method  of  management.  Time  wasted  working  on  previously  solved 
problems,  uncontrolled  schedule  slips,  inexplicable  cost  growth,  and 
development  of  items  not  needed  by  the  user  were  common  in  other 
programs.  The  founders  realized  that,  to  avoid  these  and  similar 
problems  and  to  implement  the  other  principles  they  planned  to  use 
on  this  program,  they  needed  to  manage  the  effort  tightly. 

8.  Understand  What  Is  Really  Required  —  A  key  failing  of 
many  acquisition  programs  is  changing  requirements.  The  most 
palatable  reason  is  a  change  in  threat.  To  counter  the  new  threat,  the 
program  office  must  change  the  weapon.  However,  other  reasons  for  a 
change  in  requirements  can  be  much  less  palatable.  The  program’s 
management,  which  had  seen  these  problems  before,  wanted  to  make 
sure  that  the  ATF’s  requirements  matched  user  requirements  and 
that  the  users  xmderstood  the  impact,  including  the  cost,  of  changes 
they  were  asking  for.  For  this  imderstanding  to  develop,  they  foxmd  it 
was  important  to  involve  everyone  who  had  a  stake  in  the  outcome  of 
the  fighter  in  the  requirements  process. 

Tactical  Air  Command  personnel,  as  the  operational  users,  estab¬ 
lished  the  requirements.  They  worked  their  requirements  iteratively 
with  the  program  office  to  understand  the  impact  on  the  cost,  sched¬ 
ule,  and  performance  of  the  fighter  and  then  evaluated  the  opera¬ 
tional  benefit  of  these  revised  requirements  (a  disciplined,  “bang-for- 
your-buck”  analysis).  The  ATF  program  office  also  foimd  it  helpful  to 
have  the  requirements  coordinated  at  a  level  in  the  Air  Force  high 
enough  that  any  change  would  reqmre  an  argument  good  enough  to 
sway  the  opinion  of  a  senior  decisionmaker  (the  Chief  of  Staff  of  the 
Air  Force).  See  chapter  5  for  a  discussion  of  the  definition  sind  refine¬ 
ment  of  the  ATF  requirements. 

9.  Set  Realistic  Expectations  and  Meet  Them  —  In  talking 
to  a  group  of  budding  program  managers.  General  Fain  explained, 
“The  definition  of  a  successful  program  is  a  program  that  never 
surprises  the  Beltway — ^you  cannot  allow  mis-expectations!”  The 
founders  knew  that  the  definition  of  success  depended  on  (a)  the 
individual  judging  you,  EUid  (b)  that  individual’s  expectation  of  what 
you  would  deliver. 

As  an  example,  in  the  General  Officer  Review  the  18  flag  officers 
had  the  chance  to  see  what  the  program  office  was  asking  for  in  the 
EMD  phase.  This  session  set  their  expectations  of  what  the  ATF 
program  would  and  would  not  do.  For  example,  the  expectation  that 
the  ATF  was  to  be  a  single-engine  aircraft  would  be  reset  as  they 
learned  that  both  team’s  concepts  called  for  two  engines.  If  they 
believed  the  ATF  would  fly  with  engine-thrust  reversers  (originally 
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planned  at  the  start  of  demonstration/validation),  they  would  learn 
that  the  user  and  the  program  office  dropped  reversers  for  reasons  of 
reliability,  cost,  weight,  and  complexity.  The  power  of  the  General 
Officer  Review  was  that  it  gave  these  key  individuals  the  opportunity 
to  voice  disagreement  with  the  plan  and  adjust  the  program’s  project¬ 
ed  course.  Because  the  General  Officers  all  saw  the  projected  program 
at  the  same  time,  they  all  shared  the  same  view.  Their  expectations 
(by  design)  matched  those  of  the  program  office. 

The  second  part  of  the  principle,  meet  expectations,  has  an  impor¬ 
tant  corollary.  An  individual  who  cannot  do  what  was  promised  needs 
to  explain  the  reason  to  the  correct  people  and  readjust  their  expec¬ 
tations.  Timeliness  is  critical.  As  soon  as  a  team  member  knows  an 
event  will  not  follow  the  original  schedule,  or  that  a  performance 
point  will  not  be  met,  that  individual  must  reset  everyone’s  expecta¬ 
tions  as  fast  as  possible. 

10.  Provide  Realistic  Options  —  Life  in  a  program  office  is 
spent  answering  questions.  Frequently  Pentagon  program  element 
monitors  (the  staff  who  budget  the  necessary  funds)  call  the  program 
office  to  find  out  the  impact  of  a  proposed  change  in  future  funding. 
Typically  such  questions  need  answers  in  a  short  time  (minutes, 
hours,  or  days).  These  “budget  drills”  or  “what-if  exercises”  often 
become  reality,  resulting  in  a  change  in  the  program’s  expected 
funding  profile.  The  F-22  team  policy  was  to  answer  every  question 
with  multiple  (generally  three)  accurate,  executable  courses  of 
action.  Each  answer  had  to  explain  the  impact  on  cost,  in  both 
the  affected  years  and  over  the  total  life  cycle;  on  schedule;  on  the 
performance  of  the  aircraft;  and  on  opportunity  costs.  The  program 
leadership  found  this  approach  critical  to  maintaining  its  credibility 
inside  the  Pentagon  and  helped  the  program  office  meet  realistic 
expectations  with  the  Air  Force  Headquarters  and  the  user,  Tactical 
Air  Command. 

11.  Take  a  Long-Term  View  —  Historically,  program  directors 
have  occasionally  caused  problems  by  basing  decisions  on  short¬ 
term  expedients  and  failing  to  take  into  account  long-term  conse¬ 
quences.  For  example,  the  B-IB  program’s  decision  to  delay  develop¬ 
ing  support  equipment  hurt  the  initial  readiness  of  the  aircraft. 
Deputy  Program  Director  Graves  and  the  rest  of  the  F-22  team  vowed 
not  to  let  that  kind  of  problem  happen  in  their  program.  In  resolving 
all  problems,  the  program  office  weighs  how  the  solution  will,  in  the 
long  term,  affect  the  operational  user.  The  team  calls  this  life-cycle 
decisionmaking. 
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Lockheed  demonstrated  this  concept  early  in  the  design  of  the 
production  aircraft  when  the  fundamental  design  issue  arose  of  how  to 
attach  the  wings  to  the  fuselage.  One  method  would  attach  the  wing 
stimdily  and  allow  it  to  be  installed  and  replaced  relatively  easily.  The 
second  method  wovild  keep  the  wing  attached  equally  well  but  would 
make  the  wing  harder  to  install  and  much  more  difficult  to  replace. 
The  second  type  of  wing  was  cheaper  to  manufacture  and  would  yield 
a  lighter  airplane  (both  good  news  to  a  program  director  at  the  start  of 
EMD).  The  team  opted  for  the  more  expensive  (at  least  initially)  and 
heavier  attachment  method  because  the  life-cycle  decision  clearly 
highlighted  the  problems  the  user  would  have,  should  a  wing  need 
replacement.  If  the  program  director  had  chosen  Option  2,  he  would 
have  committed  the  Air  Force  to  a  complex,  and  thus  expensive,  wing 
replacement  process.  The  decision  chose  the  method  that  cost  more  in 
the  short  term  but  would  be  much  cheaper  in  the  long  term. 

12.  Do  It  Right  the  First  Time  —  At  the  start  of  the  demon¬ 
stration/validation  phase,  Lieutenant  General  Thomas  H.  McMul¬ 
len,  who  commanded  the  Aeronautical  Systems  Division  (the  higher 
headquarters  of  the  ATF  program  office)  was  adamant  about  elimi¬ 
nating  inefficiency  in  programs  and  improving  the  way  the 
Government  developed  weapons.  He  wanted  his  people  to  accom¬ 
plish  a  task  correctly  on  the  first  try.  To  do  otherwise  meant  coming 
back  and  redoing  the  task — clearly  a  waste  of  time  and  money.  Thus, 
clearly  guided  by  McMullen  and  Fred  Rail,  his  director  of  Engineer¬ 
ing,  and  reinforced  by  lessons  learned  from  past  programs,  Fain  and 
Abell  set  up  ATF  program  operations  with  the  power  and  procedures 
to  “do  it  right”  on  the  first  pass. 

Fain  redoubled  this  effort  toward  the  end  of  demonstration/ 
validation,  when  the  Pentagon  directed  a  cost-type  contract  for  the 
EMD  phase.  He  was  concerned  that  there  would  not  be  enough 
money  in  the  budget  to  make  errors  on  the  first  pass  and  accomplish 
a  task  on  the  second  try.  As  the  EMD  phase  of  the  F-22  program 
began,  he  reiterated  that  all  members  of  the  F-22  team  were  to  “do 
it  right  the  first  time.” 

13.  Have  What  You  Need  for  the  Effort  — Abell  did  not  want 
to  start  development  of  the  aircraft  until  he  knew  what  would 
work  and  what  wouldn’t.  Having  seen  other  programs  work  hard  to 
discover  or  invent  something  before  they  could  put  their  system  into 
production,  he  knew  that  chasing  technology  could  prove  expensive 
in  both  time  and  money  and  resolved  not  to  let  the  F-22  be  held 
hostage  to  unprogrammed  advancements.  The  ATF’s  primary  focus 
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in  demonstration/validation  was  to  refine  requirements  and  mature 
technology  for  the  EMD  phase  of  the  program.  Only  if  the  contrac¬ 
tors  could  demonstrate  a  technology  would  Abell  allow  them  to  plan 
for  its  use  in  the  development  and  production  of  the  fighter. 

14.  Ensure  Everyone  Knows  What  It  Takes  to  Meet  the 
Goal  —  A  powerful  advantage  of  having  a  tightly  run,  highly  integrat¬ 
ed  program  structure  is  that  the  Program  Director  has  better  insight 
into  how  much  money  and  time  the  contractor  needs  to  complete  a  pro¬ 
gram.  With  a  firm  understanding  of  the  expected  costs,  the  program’s 
leadership  could  credibly  defend  the  F-22  budget.  They  made  sure  the 
acquisition  leadership  in  the  Pentagon  as  well  as  the  leadership  at 
Tactical  Air  Command  (which  requests  the  program’s  funding)  knew 
what  resources  the  program  needed  to  be  successful.  Providing  credible 
cost  and  schedule  requirements  to  senior  decisionmakers  helped  ensure 
that  the  F-22  program  received  its  budgeted  funding. 

15.  Use  an  Event-Based  Schedule  with  Defined  Success 
Criteria  —  This  principle  is  the  backbone  of  the  F-22  management 
system,  the  integrated  master  plan  (discussed  in  detail  in  chapter  8). 
While  working  on  other  programs,  the  F-22  management  team  had 
participated  in  many  preliminary  and  critical  design  reviews  that 
were  held  precisely  on  their  scheduled  date,  even  though  the  contrac¬ 
tor  and  the  program  office  were  not  ready.  A  review  like  this  would 
generate  many  open-action  items  that  had  to  be  completed.  In  some 
cases,  a  second  or  third  review  took  place  months  later.  As  the  F-22 
leadership  knew,  holding  a  critical  review  before  everything  is  ready 
yields  items  that  must  be  scrapped  and  reworked  (clearly  violating 
Principle  12,  do  it  right  the  first  time).  As  a  solution,  the  management 
team  devised  a  method  in  which  the  contractor  defined  intermediate 
checkpoints  in  the  program  with  criteria  for  when  an  activity  could 
be  considered  complete.  The  contractor  sequentially  arranged  these 
events  to  establish  an  event-based  schedule  that  clearly  showed  what 
had  to  be  done  and  when.  By  instilling  the  concept  that  the  F-22  would 
be  an  event-based  program  (i.e.,  the  program  wouldn’t  proceed  until 
the  event  had  successfully  met  its  success  criteria),  the  foimding 
fathers  helped  ensure  that  major  reviews  would  take  place  at  the 
correct  time — when  the  work  was  done  and  the  program  was  ready  to 
move  on  to  the  next  step. 

16.  Define  Success  and  Be  Able  to  Measure  It  —  In  general, 
people  want  to  do  a  good  job,  but  frequently  they  don’t  know  what  job 
needs  to  be  done,  or  they  do  not  find  out  how  they  are  doing  so  they 
can  improve.  The  leaders  of  the  program  must  define  what  needs  to  be 
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done,  and  team  members  must  know  the  leader^s  view  of  success.  The 
F-22  program  leadership  beheved  that  any  activity  that  could  be 
measured  could  also  be  improved.  They  envisioned  this  measure¬ 
ment  process  to  be  routine  and  standard,  a  normal  part  of  doing  busi¬ 
ness.  Their  goal  was  to  measure  and  regularly  track  every  major  part 
of  the  F-22  program  and  focus  on  areas  that  exceeded  certain  prede¬ 
termined  limits.  Fain  and  his  directors  defined  these  limits  based 
on  their  agreed-on  measures  of  success — technical  performance  mea¬ 
sures  (see  chapter  8).  Significantly,  they  chose  the  success  criteria 
before  the  event. 

17.  Reward  Success  —  The  F-22  program  has  rewarded  success 
at  many  levels.  The  current  development  effort  uses  a  cost-plus- 
award-fee  contract  in  which  the  Government  awards  the  contractor  its 
profit  every  6  months  based  on  how  well  it  has  performed.  On  a  small¬ 
er  scale,  every  Friday  the  program  office  picks  the  outstanding  indi¬ 
vidual  or  team  for  that  week.  An  old  adage  says  that  success  breeds 
success,  and  a  key  to  making  that  adage  come  true  is  identifying  and 
acknowledging  success.  Having  a  standard  measurement  process  that 
eaisily  enables  the  leadership  to  evaluate  program  performance  also 
enables  the  leadership  to  fairly  identify  success.  Recognition  serves  to 
reinforce  positive  behavior  by  providing  positive  feedback  to  the  win¬ 
ners  and  to  all  other  members  of  the  team. 

18.  Focus  on  a  Win-Win  Relationship  —  A  popular  program 
office  saying  was,  “If  the  contractor  wins,  the  Government  wins.  If 
the  contractor  loses,  the  Government  loses.”  In  the  past,  whether 
reviewing  a  specification  or  conducting  a  negotiation,  the  contrac¬ 
tors  and  the  Government  often  would  be  at  odds.  The  program  lead¬ 
ership  realized  that  the  Govemment/contractor  team  would  be  more 
effective  if  the  members  were  working  together  to  solve  their  prob¬ 
lems.  If  the  contractor  developed  a  fighter  system  that  met  the  user's 
stated  needs  within  the  planned  budget,  cost,  and  schedule,  the 
Grovernment  would  be  satisfied — it  would  win.  If  the  contractor 
delivered  the  weapon  system  as  promised,  it  would  win — that 
is,  earn  the  full  profit  (awarded  through  the  cost-plus-award-fee 
contract).  The  converse  is  true  as  well.  If  the  contractor  did  not 
deliver  a  system  that  met  the  user's  needs  or  was  over  budget,  cost, 
or  schedule,  the  Government  would  lose  because  it  wouldn't  be 
satisfied;  so  would  the  contractor,  since  its  profit  would  be  reduced. 
This  idea  encouraged  all  Government  team  members  to  join  their 
contractor  counterparts  to  form  one  unit  that  would  develop  a  fight¬ 
er  weapon  system  that  met  warfighter  needs. 
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19.  Guarantee  Open  Communication  —  In  earlier  programs, 
problems  that  were  hard  to  solve  and  expensive  to  fix  had  reached 
Fain.  When  he  delved  into  them,  he  found  that  his  people  had  known 
of  them  for  some  time  but  couldn’t  solve  them  at  their  level.  He  real¬ 
ized  that  if  he  had  known  of  the  problems  at  the  time  they  occurred, 
he  could  have  solved  them  easily  and  cheaply.  Based  on  this  observa¬ 
tion,  he  wanted  to  establish  an  environment  where  a  person  finding 
such  a  problem  could  report  it  in  a  way  that  would  help  get  the  prob¬ 
lem  solved.  Setting  up  a  win-win  relationship  resulted  in  common 
trust  and  understanding  and  established  the  environment  that  Fain 
was  looking  for,  one  that  supports  open  communication. 

20.  Achieve  Success  with  a  Positive  Attitude  and  Focus  — 

Another  popular  saying  in  the  F-22  program  office  was  'The  key  to 
success  is  positive  attitude  and  focus.”  A  team  of  people  focused  on  the 
correct  objective  and  with  the  right  frame  of  mind  could  tackle  any 
problem.  The  program’s  leadership  ensiired  that  the  team  stayed 
aimed  in  the  right  direction  and  at  the  right  issues.  They  refused  to 
let  the  team  complain  or  worry  about  items  over  which  they  had  no 
control  (summed  up  with  two  other  popular  program  office  expres¬ 
sions:  "no  whining”  and  “no  hand  wringing”). 

Summary 

The  20  principles  of  acquisition  in  Figure  2-1  don’t  seem  very 
revolutionary.  They  are  simple  and  straightforward,  bom  of  common 
sense.  However,  they  are  effective.  In  some  organizations,  they  also 
are  fairly  rare.  How  many  of  these  principles  have  been  violated  in 
other  organizations?  How  effective  could  those  organizations  have 
been  if  they  had  implemented  all  20?  These  principles,  which  apply  to 
almost  any  other  acquisition  program  (and  to  most  other  situations  as 
well),  serve  as  the  F-22  program’s  foundation.  The  two  concepts  in 
chapter  3  complete  the  basic  acquisition  truths. 


n^cs  courtesy  of  Lwskheed-Martifi. 

The  fine  lines  of  the  YF-22  are  displayed  in  this  head-on  view.  The  aircraft  was  designed  for  a  reduced  ladar  signature  as  well  as 
the  ability  to  cruise  at  supersonic  sp^s. 
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Concurrent  leadership  is  a  new  way  of  doing  business.  It  is  a 
new  culture  that  relies  on  successful  teaming.  We  have  to  have 
the  right  people.  Law-and-order  managers  no  longer  cut  it.  Our 
team  members  must  be  functional  experts  but  with  a  broader 
perspective.  They  must  have  a  program  view. 

James  (Micky)  Blackwell, 
Lockheed  F-22  Program  Director 


Chapter  2  described  the  principles  that  form  the  foundation  of 
the  F-22  program.  Expanding  on  these  principles,  the  program 
founders  embodied  them  in  two  central  methods  they  used  to 
manage  the  program.  Serving  as  central  themes  for  developing  the 
program,  one  method  centered  on  how  to  achieve  program  success  by 
managing  risk,  the  other  on  how  to  empower  one’s  work  force  through 
concurrent  leadership. 


Risk  Management 

All  weapon  development  programs  seek  to  design  and  deliver  a 
product  that  meets  the  operational  user’s  needs  within  allocated 
resources  (both  time  and  money).  From  experience  with  earlier  pro¬ 
grams,  the  management  team  knew  that  the  chief  risks  to  successful 
performance  lay  in  three  areas.  One  area  relates  to  problems  stem¬ 
ming  from  lack  of  firm  requirements  or  from  misunderstood  require¬ 
ments.  Another  relates  to  problems  stemming  from  immature  or  miss¬ 
ing  technologies.  Both  sets  of  problems  negatively  affect  cost,  schedule 
and  performance.  But  after  controlling  these  problem,  programs  can 
still  get  into  trouble.  A  third  area  of  risk  centers  on  inadequacies  in 
planning  and  tracking  the  overall  effort. 

From  this  assessment,  the  program  office  evolved  the  three¬ 
pronged  approach  to  managing  risk: 

1.  Refine  and  understand  the  requirements  of  the  program 

2.  Manage  the  needed  technologies 

3.  Use  a  product-based  management  system. 

The  ATF  program  implementation  of  these  ideas  is  described  in 
chapters  5  (requirements),  6  (technology),  and  7  and  8  (the  manage¬ 
ment  system).  Figure  3-1  serves  as  a  “road  map”  of  risk  management. 


23 


24  Acquisition  for  the  21st  Century 


Refine  and  Understand  the  Requirements 

It’s  a  simple  enough  concept:  build  what  the  user  needs.  But 
program  manager  after  program  manager  has  foimd  that  doing  so  is  not 
simple  at  all.  The  ATF  foimders  discovered  that  they  had  to  get  all  of  the 
key  players  to  agree  on  what  was  really  required  and  on  what  they 
wanted  to  achieve.  Several  of  the  ATF/F-22  principles  described  in  chap¬ 
ter  2  apply  to  requirements  definition  and  refinement: 

4.  Involve  everyone 

6.  Ensure  ownership 

7.  Use  a  disciplined  approach 

8.  Understand  what  is  really  required. 

Involve  everyone  —  Experience  on  the  ATF  program  showed 
that  the  users  will  adjust  their  requirements  if  they  imderstand  the 
impact  (for  example,  on  the  cost,  reliability,  or  maintenance  capabili¬ 
ty)  of  those  requirements.  To  do  that,  they  had  to  be  fully  involved. 
Likewise,  if  contractors  understand  why  a  user  has  a  certain  require¬ 
ment,  they  can  generate  innovative  ways  to  meet  it.  This  process 
works  best  if  all  of  the  players  (program  office,  user,  contractors)  are 
mutually  involved  and  able  to  talk  to  each  other. 


Figure  3-1.  Elements  of  Risk  Management 
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Ensure  ownership  —  As  mentioned  in  chapter  2,  people  will 
commit  to  making  something  happen  if  they  have  bought  into  the 
concept — if  they  have  ownership.  Bringing  all  of  the  players  togeth¬ 
er  to  define  and  refine  the  system’s  requirements  gives  them  a 
vote  in  what  is  finally  decided,  which  helps  guarantee  they  will  all 
support  those  requirements. 

Use  a  disciplined  approach  —  Reviews  on  a  regular  basis 
force  all  parties — program  office,  users,  and  contractors — to  read¬ 
dress  the  user’s  needs.  The  users  revalidate  their  requirements.  The 
contractors  reaffirm  their  ability  to  meet  them.  If  the  weapon  system 
won’t  meet  a  user  requirement,  the  program  must  either  fix  the 
weapon  (if  possible)  or  amend  the  requirement.  To  adjust  the 
requirement,  the  team,  led  by  the  user,  must  reexamine  the  weapon, 
threat,  and  other  factors  driving  the  requirement)  to  see  if  the 
requirement  for  change  is  valid  or  if  it  can  be  met  in  some  other  way 
(such  as  through  a  change  in  tactics).  If  there  is  a  mismatch  between 
the  weapon  system  and  the  user’s  requirements,  it  is  best  to  know 
early  so  the  service  can  change  one  or  the  other. 

Understand  what  is  really  required  —  Clear  communication 
with  the  group  that  needs  the  product  and  a  clear  understanding 
within  the  user  group  of  what  it  wants  helps  to  ensure  a  successful 
program.  By  working  as  an  integrated  team  and  regularly  reviewing 
the  user’s  requirements  and  the  contractor’s  ability  to  meet  them,  the 
program  will  establish  a  solid,  achievable  set  of  requirements,  will 
greatly  reduce  scrapping  and  reworking  of  items,  and  will  help  ensure 
that  the  user  gets  a  weapon  that  does  what  it  needs  to  do. 

Manage  the  Needed  Technologies 

The  starting  assumption  behind  this  strategy  is  that  the  new 
system  requires  new  technology.  The  F-22,  a  supersonic,  highly 
maneuverable  stealth  fighter,  clearly  did.  In  other  programs,  such  as 
the  Army  RAH-66  Comanche  helicopter,  the  Navy  SSN-21  Seawolf 
attack  submarine,  or  the  Marine  V-22  Osprey  tilt-rotor  aircraft,  the 
need  for  new  technology  is  also  obvious.  However,  the  need  for  tech¬ 
nology  does  not  guarantee  its  availability.  The  program  must  focus  on 
developing  and  proving  the  technology  that  is  needed  and  appropriate 
to  support  the  user’s  requirements.  Given  the  current  lengthy  devel¬ 
opment  cycles,  developing  and  proving  technology  at  the  beginning  of 
the  program  may  mean  entering  production  with  something  other 
than  the  latest  technology.  What  is  important  is  to  enter  into  produc¬ 
tion  not  with  the  latest  technology  but  with  technology  that  works  to 
meet  the  user’s  needs. 
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The  ATF/F-22  principles  that  apply  to  technology  development 
and  definition  are: 

9.  Set  realistic  expectations  and  meet  them 

13,  Have  what  you  need  for  the  effort 

14,  Ensure  everyone  knows  what  it  takes  to  meet  the  gocU 

19,  Guarantee  open  communication. 

Set  realistic  expectations  and  meet  them  —  The  Government 
relies  on  the  contractor  to  develop  a  weapon  to  meet  the  user’s 
requirements  and  deliver  the  weapon  with  the  technology  necessary 
to  meet  those  requirements.  In  the  past,  some  programs  have  oversold 
what  new  technology  could  do.  Early  promises  served  to  set  the  user’s 
expectations  at  an  unreasonably  high  level,  and  the  new  weapon, 
although  better  than  previous  systems,  may  not  have  met  the  user’s 
original  expectations.  Understandably,  this  resulted  in  dissatisfaction. 
Involving  the  user  in  proving  what  technology  can  actually  do  greatly 
helps  in  setting  appropriate  expectations. 

Have  what  you  need  for  the  effort  —  As  described  in  chapter 
2,  the  program  office  wanted  to  make  sure  that  the  technology 
promised  for  ATF  problems  actually  existed  and  worked.  To  bank  on 
the  successful  application  of  an  unproven  technology  would  greatly 
increase  the  risk  in  the  program.  The  program  had  to  prove  the  tech¬ 
nology  through  analysis,  demonstrations,  and  other  testing. 

Ensure  everyone  knows  what  it  takes  to  meet  the  goal, 
and  guarantee  open  communication  —  During  the  demonstra¬ 
tion/validation  phase,  the  ATF  development  teams  consisted  of  many 
different  contractors.  Each  contractor  had  its  own  technology  devel¬ 
opment  laboratories  staffed  by  hundreds  of  scientists  and  engineers 
knowledgeable  about  different  materials  and  approaches  applicable 
to  challenges  of  designing  a  new  fighter  aircraft.  Because  the  lead 
contractors  (Lockheed  and  Northrop)  clearly  communicated  the 
challenges  and  the  approaches  to  each  member  of  their  team,  each 
competing  team  could  take  advantage  of  a  large  body  of  knowledge, 
which  solved  many  demanding  problems.  Whether  an  effort  involves 
many  contractors  or  just  many  people,  open  communication  and  clear 
statements  of  what  it  takes  to  meet  the  goals  draws  in  everyone  to 
help  develop  the  needed  technology. 

Use  a  Product-Based  Management  System 

Exceeding  planned  costs,  not  staying  on  schedule,  and  not  meet¬ 
ing  requirements  has  historically  attracted  a  great  deal  of  attention. 
As  mentioned  in  chapter  1,  during  the  1950s  and  1960s  many 
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programs  had  large  cost  overruns  and  schedule  slips.  Although  the 
Department  of  Defense  (DOD)  has  made  large  improvements  in  this 
area  (a  recent  study  shows  the  DOD  controls  costs  much  better  than 
most  other  government  agencies),  tight  management  is  essential  to 
keep  cost,  schedule,  and  technical  performance  under  control.  The 
ATF  management  team  wanted  every  person  on  the  F-22  program  to 
be  as  effective  as  possible.  Workers  at  all  levels  needed  to  feel  a 
strong  commitment  to  their  task  and  a  sense  of  pride  and  ownership 
in  what  they  did  for  the  F-22.  As  a  result,  the  management  team 
wanted  to  develop  a  management  system  that  focused  workers  at  all 
levels  on  their  product. 

The  key  management  system  ATF/F-22  principles  are: 

5.  Integrate  the  entire  system 

6.  Ensure  ownership 

7.  Use  a  disciplined  approach 

8.  Understand  what  is  really  required 

11.  Take  a  long-term  view 

13.  Have  what  you  need  for  the  effort 

14.  Ensure  everyone  knows  what  it  takes  to  meet  the  gofd 

15.  Use  an  event-based  schedule  with  defined  success 
criteria 

16.  Define  success  and  be  able  to  measure  it 

17.  Reward  success. 

The  length  and  depth  of  this  list  highlight  the  importance  of  the 
management  system.  To  manage  cost,  schedule,  and  performance  and 
follow  the  principles  above,  the  F-22  program  developed  several 
management  tools  and  procedures  (discussed  in  chapter  8).  These 
include  an  event-based  schedule  coupled  with  success  criteria  and 
measures  of  effectiveness,  and  a  means  to  track  performance,  hold 
individuals  accountable,  and  reward  the  performance  of  the  team. 
The  F-22  program  organization  empowered  the  lowest  tier  members 
of  the  team  while  it  also  ensured  a  well-balanced  approach  to  the 
product  (see  chapter  7). 

Integrate  the  entire  system  —  The  F-22  program  introduced 
this  powerful  idea  to  the  DOD.  In  the  past,  management  teams  used 
separate  tools  to  look  at  cost,  schedule,  and  technical  performance.  To 
be  successful,  the  management  system  must  tie  together  existing  or 
newly  created  tools  so  that  the  management  team  can  assess  all 
aspects  of  performance.  The  most  important  benefit  of  such  an 
approach  is  clarity  of  information.  Management  can  easily  and  clear¬ 
ly  see  how  the  program  is  performing  from  the  integrated  views  of 


28  Acquisition  for  the  21  st  Century _ 

cost,  schedule,  and  performance.  An  integrated  management  system 
greatly  improves  team  member — especially  the  program  director’s — 
situational  awareness  of  the  program. 

Ensure  ownership  —  The  management  system  must  be  struc¬ 
tured  in  such  a  manner  that  all  members  of  the  team,  from  program 
manager  to  line  draftsman,  believe  the  product  they  are  working  on 
is  their  personal  responsibihty.  One  way  to  do  this  is  to  carve  the 
program  into  pieces,  units,  and  subunits  and  assign  individuals  to 
work  on  the  development  of  the  products  within  them.  The  manage¬ 
ment  system  should  hold  individuals  accountable  for  the  development 
of  their  products  and  give  them  the  authority  to  make  those  products 
work.  As  in  the  integrated  management  system  mentioned  above,  the 
system  should  hold  each  team  member,  regardless  of  technical  special¬ 
ty,  accountable  for  the  cost,  schedule,  and  performance  of  the  product. 

Use  a  disciplined  approach  —  The  management  system  of  any 
complex  endeavor — defense  acquisition,  automobile  design  and 
production,  even  home  construction — must  plot  a  rigorous,  organized 
path  to  its  next  phase. 

Understand  what  is  really  required  —  This  idea  applies  to 
much  more  than  the  requirements  of  the  end  product.  The  team 
needs  to  think  through  all  of  the  steps  needed  to  design,  develop,  test, 
produce,  and  field  the  user’s  weapon  system.  The  management 
system  requires  and,  in  fact,  depends  on  a  detailed  up-front  plan  that 
takes  the  program  from  the  current  starting  point  to  the  next  phase. 
In  the  ATF  program,  the  competing  contractors  started  their  detailed 
planning  as  part  of  the  preparation  for  the  EMD  phase  in  the  last  2 
years  of  the  demonstration/validation  phase,  when  the  two  sets  of 
competing  teams  had  a  good  understanding  of  the  requirements  and 
their  design  solution.  However,  the  concept  applies  to  any  phase. 
What  is  important  is  to  have  a  management  system  that  forces  the 
team  to  plan  well  in  advance  yet  allows  the  flexibihty  needed  to  deal 
with  unanticipated  problems. 

Take  a  long-term  view  —  The  management  system  must  allow 
decisionmaking  to  focus  on  long-term  considerations  of  dehvering  a 
product  that  meets  the  user’s  requirements.  A  detailed,  long-range 
schedule  helps  workers  at  every  level  see  the  path  to  the  end  product. 
Tracking  current  product  performance  against  the  user’s  desired  oper¬ 
ational  (delivered)  requirements  helps  focus  team  members  on  the 
goal  of  user  satisfaction. 

Have  what  you  need  for  the  effort  —  To  look  at  all  aspects 
of  cost,  schedule,  and  technical  performance  for  all  products,  as 
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discussed  above,  the  system  needs  a  method  of  collecting  data 
that  assesses  the  status  of  each  area.  It’s  best  to  have  one  set  of  data 
that  the  entire  team  uses:  Government  team  members  should  use 
contractor  data  in  the  format  that  the  contractor  normally  uses.  In 
fact,  the  ATF  program  office  emphasized  the  efficient  use  of  data: 
’’Create  data  once — use  it  many  times." 

Using  contractor  data  in  the  contractor’s  format  saves  money  (the 
Government  doesn’t  have  to  pay  the  contractor  to  translate  informa¬ 
tion  into  a  Government-required  format).  It  reduces  communication 
errors  (bringing  realism  to  the  hackneyed  expression,  “let’s  all  sing  off 
the  same  sheet  of  music”).  Having  the  contractor,  program  office,  and 
iiser  looking  at  the  same  sheet  of  paper  with  the  same  data  encour¬ 
ages  teamwork.  The  management  system  should  get  the  necessary 
information  in  a  way  that  uses  the  contractor’s  format,  saves  time 
that  can  be  apphed  to  improving  the  user’s  product,  saves  taxpayer 
money,  and  promotes  teamwork. 

Ensure  everyone  knows  what  it  takes  to  meet  the  goal  — As 

discussed  above,  the  management  system  must  force  and  facilitate 
detailed,  up-front  planning  that  drives  the  team  to  chart  a  clear  path 
to  the  final  product.  The  management  system  must  also  require  that 
each  team  follow  one  overall  plan  so  that  each  team  member  knows 
what  activities  must  take  place  to  meet  the  final  objective  and  each 
team  xmderstands  its  relation  to  the  overall  program.  For  example,  if 
the  model  department  falls  behind  in  bmlding  wind  tunnel  test 
models,  the  wind  tiinnel  tests  are  delayed.  If  data  from  wind  tunnel 
tests  are  delayed,  aerodynamic  load  factors  for  the  airframe  design 
team  don’t  arrive  in  time.  If  the  design  of  the  airframe  is  delayed,  the 
construction  of  the  first  flying  prototype  will  also  be  delayed.  If  the 
model  makers  can  look  at  a  single  schedule  and  see  the  impact  of  their 
delays  on  their  ultimate  product  (the  flying  prototype  and,  ultimately, 
the  operational  airplane),  they  will  have  a  stronger  motivation  to  find 
a  way  to  get  back  on  schedule. 

Use  an  event-based  schedule  with  defined  success  criteria  — 

The  management  system  must  have  a  plan  that  shows  interme¬ 
diate  events  that  need  to  occur  to  arrive  at  the  final  product  and  in 
what  sequence  they  must  happen.  The  plan  must  also  define  what  con¬ 
stitutes  successful  completion  of  each  event.  Finally,  the  management 
system  must  tie  the  events  to  a  calendar  so  that  the  program  can  be 
costed  (to  determine  how  much  effort  by  year  and  thus  how  much 
money  by  year  is  necessary)  and  scheduled  (to  reserve  test  ranges 
and  other  facilities  as  well  as  schedule  deliveries  for  materials). 
Even  though  events  (for  instance,  budget  reductions)  may  delay  the 
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program,  the  sequence  usually  does  not  change  (the  model  maker 
must  make  the  model  before  the  wind  tunnel  can  collect  data,  but  it  can 
happen  in  May  instead  of  January). 

A  preapproved  definition  of  success  is  important  to  support 
the  original  planning.  For  example,  the  airframe  design  team 
might  determine  it  needs  aerodynamic  loads  for  the  aircraft  with 
flaps  deployed  and  retracted,  calling  for  two  models.  At  the  start  of 
the  program,  the  design  team  would  define  successful  completion  of 
wind  tunnel  testing  as  capturing  data  for  an  aircraft  with  flaps 
deployed  and  retracted.  If,  during  the  course  of  the  program,  the 
model  maker  fell  behind  and,  to  catch  up,  built  a  model  that  only 
simulated  flaps  retracted,  the  event  of  wind  tunnel  testing  could  not 
be  completed  because  it  requires  data  for  both  configurations.  With  a 
defined  criterion  for  wind  tunnel  testing,  the  team  focuses  on  captur¬ 
ing  the  data  needed  for  completing  the  event,  rather  than  meeting 
the  schedule.  In  this  manner,  the  management  system  encourages 
activities  to  support  the  quality  of  the  product  (in  this  example,  a 
complete  design  using  data  for  both  flap  configurations). 

Emphasizing  the  quality  of  the  product  does  not  imply  that  the 
schedule  is  not  important.  As  described  above,  the  program  holds  each 
team  accountable  for  meeting  all  aspects  of  performance,  including 
schedule  performance.  However,  unlike  a  calendar-based  program 
schedule,  the  event-based  schedule  focuses  on  achieving  the  prede¬ 
fined  activities  according  to  the  predefined  criteria.  Other  portions  of 
the  management  system  (such  as  the  award  component)  can  motivate 
successful  schedule  performance.  The  key  is  not  to  trade  product 
quahty  for  schedule  performance. 

Some  may  fear  that  up-front  planning  locks  the  team  into  an 
unchangeable  path.  But  new  events  (such  as  a  destroyed  wind  tunnel 
or,  perhaps,  an  improved,  validated  computational  fluid-dynamics 
computer  model  that  successfully  predicts  aerodynamic  performance) 
will  force  a  change  in  the  sequence,  success  criteria,  or  event  timing 
of  the  plan.  If  this  occurs,  it  is  important  to  change  the  plan  in  an 
integrated  manner.  If  all  players  on  the  team  (the  model  makers,  the 
wind  tunnel  technicians,  the  airframe  designers,  and  the  flight  test 
team)  agree  to  the  change  and  understand  its  impact,  then  the  plan 
must  be  changed. 

Define  success  and  measure  it  —  For  a  management  system 
to  hold  an  individual  responsible  for  a  product  (really  a  subproduct  of 
the  final  product  delivered  to  the  customer),  the  system  must  deter¬ 
mine  how  well  that  product  is  progressing  compared  to  the  projected 
targets  for  cost,  schedule,  and  technical  performance.  First,  the 
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system  must  define  success — that  is,  define  the  product  requirements 
in  terms  of  cost,  schedule,  and  technical  performance.  It's  important 
to  note  that  requirements  for  lower  tier  products  (subproducts)  are 
based  on  performance  targets  for  higher  level  products  (for  example, 
the  cost,  schedule,  and  performance  requirements  of  the  cockpit  must 
support  the  cost,  schedule,  and  performance  requirements  of  the 
overall  airframe),  which  tie  into  the  overall,  top-level  product 
requirements.  The  management  system  must  track  how  the  product 
is  meeting  its  targets  at  each  level. 

Reward  success  —  With  success  defined  and  measured,  the  next 
step  is  for  the  management  system  to  reward  members  (both  individ¬ 
uals  and  teams)  for  their  performance.  The  rewards  will  vary.  The 
challenge  for  the  program  leadership  (both  the  contractor  and  the 
Government)  is  to  develop  rewards  that  motivate  future  high-level 
performance.  The  contractor,  as  a  company  or  group  of  companies, 
can  be  rewarded  financially  through  an  award-fee  or  incentive-fee 
contract.  The  contractor  could  choose  to  allocate  this  award  to  key 
performers  responsible  for  the  team's  success.  Or  the  Government  and 
contractor  could  recognize  team  members  through  a  team  awards 
program  (for  example.  Outstanding  Team  of  the  Month,  Best  Team 
Member  of  the  Quarter).  Whatever  methods  one  chooses,  the  key  is  to 
recognize  and  reward  the  performance  of  both  individuals  and  teams. 

Concurrent  Leadership 

Leaders  know  that  mission  success  depends  upon  one's  people. 
The  challenge  is  to  structure  the  organization  so  that  people  can 
accomplish  the  mission.  Fain  built  upon  pre-1990  Air  Force  IPD 
concepts  by  applying  concurrent  leadership,  empowering  teams  to 
develop  integrated  solutions.  As  described  in  chapter  7,  concurrent 
leadership  is  an  essential  element  of  the  F-22  program  management 
system.  Some  say  it's  the  secret  to  the  program's  success. 

General  Fain  explained  the  genesis  of  concurrent  leadership 
during  an  interview  with  Donald  Fujii  of  the  Defense  Systems 
Management  College  in  July  1991,  just  before  the  start  of  the  EMD 
phase  of  the  F-22  program: 

Fain:  This  is  my  equation  for  how  to  run  a  program.  When  I  have  a 
problem  I  get  together  the  functional  experts  from  contracts, 
finance,  engineering,  management,  and  test  (if  I  need  him).  I 
sit  them  around  this  table;  they’re  called  the  FOG  [front  office 
group].  I  say,  “This  is  what  we  need.  Generate  it.”  Each  one  of 
these  guys  has  the  capability  to  walk  out  of  their  functional 
stovepipe  and  generate  a  solution  based  on  his  expertise,  but 
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in  a  general  way,  taking  into  account  the  others*  perspectives 
and  constraints.  When  we  first  started  this  I  got  a  strategy,  for 
example,  based  on  contracts,  but  it  was  a  general  strategy. 
After  a  couple  of  years  of  doing  this,  I  started  getting  strate¬ 
gies  that  weren’t  based  on  contracts  but  were  based  on  every¬ 
thing  (engineering,  finance,  management,  test,  and  contracts). 
That  happened  because  they  all  sat  around  this  table  and 
watched  each  other  and  learned  each  other's  functional  con¬ 
straints,  issues,  and  limitations. 

Fujii:  They  were  thinking  in  an  integrated  fashion. 

Fain:  Right,  right,  right!  That's  the  concept.  [It  is  important]  to  have 
the  functional  expert  deal  with  the  functional  raw  data.  For 
example,  I  don’t  know  everything  I  need  to  know  about  the 
contractual  raw  data.  But  I've  got  someone  who  does.  But  I 
won't  let  him  answer  a  question  from  a  contract  point  of  view. 
I  let  him  answer  it  from  a  general  point  of  view....  That  is  the 
secret  to  how  we  have  run  the  ATF  for  the  last  3  or  4  years.  It 
has  been  very  successful. . . . 

After  I  bring  them  into  this  room,  when  we  get  done  sort¬ 
ing  out  the  issue,  we  have  a  strategy  that  covers  all  aspects, 
an  integrated  approach.  And  then,  you  would  be  surprised 
how  easy  it  [the  solution]  is  to  implement. 

Fujii:  You  are  sort  of  doing  management  concurrency.  Like  the 
design  concurrency  notion  where  you  get  designers  and  engi¬ 
neers  and  manufacturing  all  together. 

Fain:  Concurrent  engineering?  Yes.  We  are  doing  concurrent 
management.  No,  make  that  concurrent  leadership. 

The  notion  of  concurrent  leadership  may  be  best  shown  by 
comparing  three  models  of  management — the  traditional  method  of 
management,  the  model  used  during  the  ATF  demonstration/valida¬ 
tion  phase  described  above,  and  the  fully  developed  F-22  concurrent 
leadership  model. 

Traditional  Management 

Organizations  with  traditional  program  management  (shown  in 
Figure  3-2)  tend  to  be  focused  along  functional  lines.  Each  specialty 
resides  in  its  respective  functional  area,  or  stovepipe.  Structural 
engineers,  avionics  engineers,  and  support  equipment  engineers  all 
work  in  the  engineering  division.  Maniifacturing  experts  work  in  the 
manufacturing  division.  Finance  and  contracting  professionals  work 
in  their  respective  departments. 

In  the  traditional  model,  experts  tend  to  rise  to  ever-higher 
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management  positions  in  their  functional  stovepipes.  As  they  move  up 
in  the  organization,  they  tend  to  see  more  of  other  elements  of  the  orga¬ 
nization,  gaining  a  perspective  that  allows  them  to  deal  with  broader 
issues.  When  a  problem  surfaces  in  an  organization,  the  leader  goes  to 
the  functional  chiefs  and  requests  data.  The  appropriate  speciahsts 
generate  raw  data,  and  the  functional  chiefs,  with  their  broader  per¬ 
spective,  make  sure  the  data  are  in  a  form  that  the  leader  can  xise.  The 
functional  chiefs  then  present  the  data  to  the  leader.  The  leader  inte¬ 
grates  the  data  and,  alone  or  with  the  functional  chiefs,  decides  on  a 
strategy,  implements  a  course  of  action,  and  produces  a  solution. 

Admittedly,  the  example  above  is  simplistic,  but  it  serves  to  point 
out  that  in  a  traditional  organization,  the  burdens  of  data  integration 
and  problem  solving  tend  to  rise  to  the  leader’s  level. 

The  ATF  DemonstrationA^alidation  Phase  Leadership  Model 

As  Fain  described,  the  ATF  program  office  developed  a  different 
approach  during  the  demonstration/validation  phase.  Relying  on  the 
principles  of  involvement  and  ownership,  he  created  a  Front  Office 
Group  (FOG),  consisting  of  the  ATF  program  office  chief  functional 
experts  in  finance  (program  control),  contracts,  engineering,  manage- 


Figure  3-2.  Traditional  Management  Model 


SOURCE:  “Acquisition  Program  Leadership  for  the  1 9908,"  F-22  program  office  briefing  (Wright-Patterson  Air  Force 
Base.  OH,  November  1992). 
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ment  (projects),  and  testing.  They  formed  an  integrated  team  to 
address  problems.  This  leadership  model  (shown  in  Figure  3-3) 
differed  from  the  traditional  model  in  that  the  chief  functional 
experts  did  not  just  hand  data  to  the  leader  but  as  a  team  worked 
through  the  data  integration  process,  created  the  strategy  to  solve  the 
problems,  and  then  implemented  the  solution. 


Figure  3-3.  ATF  DemonstratiorWalidation  Phase  Leadership  Model 


Decide  on  a  Strategy  ^ 


The  F-22  Concurrent  Leadership  Model 

The  quality  of  the  solutions  that  came  out  of  FOG  meetings 
impressed  the  group.  As  the  group  members  prepared  for  the  EMD 
phase,  they  concluded  that  the  FOG’s  success  should  be  replicated  at 
lower,  subunit  levels  of  the  organization  instead  of  keeping  the  multi¬ 
functional  team  only  at  the  highest  level  of  the  Grovernment  program 
office.  The  teams  could  consist  of  program  office,  user,  and  contractor 
personnel.  Fain  dubbed  this  leadership  approach  concurrent  leader¬ 
ship,  shown  in  Figure  3-4.  (See  chapter  7  for  a  detailed  discussion  of 
the  implementation  of  concurrent  leadership  for  the  F-22  program  via 
integrated  product  teams.) 

In  concurrent  leadership,  individuals  at  the  worker/action  level 
from  different  functional  areas  work  together  to  identify  problems  and 
develop  solutions.  Their  team  focuses  on  a  specific  product,  process,  or 
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issue.  A  multidisciplinary  team  manages  an  item  based  on  a  range  of 
viewpoints,  experiences,  and  expertise.  Each  functional  area  has  its 
own  requirements,  constraints,  strengths,  and  weaknesses.  A  great 
engineering  solution  may  not  be  affordable,  and  the  finance  officer  can 
rapidly  identify  that.  A  contractual  solution  proposed  by  a  manager 
may  violate  the  Federal  Acquisition  Regulations,  which  the  contract¬ 
ing  member  of  the  team  can  point  out.  A  manufacturing  teammate  can 
refine  the  design  engineer’s  proposed  solution  to  meet  the  performance 
requirements  and  still  allow  the  part  to  be  easily  machined.  An  impor¬ 
tant  aspect  of  concurrent  leadership  is  that  the  roles  of  the  team  mem¬ 
bers  differ  from  the  roles  found  in  the  traditional  leadership  model. 


Figure  3-4.  Concurrent  Leadership  Model 
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The  role  of  the  team  leader.  Surprisingly,  the  biggest  differ¬ 
ence  between  concurrent  leadership  and  the  other  two  models  lies 
not  at  the  working  level  but  at  the  management  level.  Concurrent 
leaders  are  not  directors  but  facilitators  and  coaches.  The  leaders 
bring  in  the  right  people  and  functional  disciplines  to  help  solve 
the  problem.  They  do  not  bring  every  member  of  their  team  in  on 
every  problem,  only  those  whose  expertise  or  responsibility  is  need¬ 
ed.  Therefore,  the  team’s  emphasis  will  change  depending  on  the 
problem  to  be  addressed. 


36  Acquisition  for  the  21st  Century 


Another  major  difference  is  that  the  leader  does  not  make  the 
decisions.  To  do  that  would  take  ownership  away  from  the  team 
members.  Instead,  the  leader’s  job  is  to  make  sure  that  the  team 
makes  a  good  decision. 

The  role  of  team  members.  The  concept  of  concurrent  leader¬ 
ship  derives  its  strength  from  the  power  that  each  member  brings  to 
the  team.  Typically  an  individual  on  a  team  is  the  lone  expert  on  that 
team  in  a  particular  field.  Thus,  all  team  members  need  to  be  profi¬ 
cient  (ideally,  experienced  and  well  trained)  in  their  specialty. 
However,  the  members  must  be  able  to  grasp  issues  in  ways  that  go 
beyond  their  narrow  expertise.  As  in  the  ATF  demonstration/valida¬ 
tion  phase  leadership  model  (Figure  3-3),  members  must  work  out 
each  problem  from  their  teammates’  viewpoint.  The  design  engineer 
must  think  about  the  cockpit  canopy  from  a  maintenance  technician’s 
point  of  view.  The  program  manager  must  see  the  issue  as  the  finan¬ 
cial  member  of  the  team  would  see  it.  This  broad  perspective  is  criti¬ 
cal  to  ensure  that  a  balanced  solution  emerges.  The  final  responsibil¬ 
ity  of  team  members  in  concurrent  leadership  is  to  take  ownership  of 
each  of  the  team’s  actions  and  outcomes  and  to  actively  contribute 
their  expertise  to  the  team’s  product. 

Summary 

The  20  principles  of  acquisition  outlined  in  chapter  2  and  embod¬ 
ied  in  the  methods  used  to  manage  risk  and  to  empower  the  work 
force  through  concurrent  leadership  underlie  the  success  of  the 
ATF/F-22  program.  Part  II  describes  how  the  program  actually 
applied  these  ideas. 


Part 


Application 

The  principles  of  acquisition  embodied  in  the  methods  used  by 
the  ATF/F-22  program  to  manage  risk  and  empower  the  work 
force  apply  to  most  weapon  acquisition  programs.  Moreover, 
when  adopted,  they  will  make  future  programs  much  more  success¬ 
ful.  The  challenge  for  any  program  manager  is  to  apply  these  ideas 
in  a  way  that  makes  sense  for  the  program  and  the  environment 
in  which  it  will  operate.  Chapters  4-9  review  how  the  ATF/F-22 
program  applied  the  principles  and  methods  to  satisfy  the  opera¬ 
tor’s  need  for  a  weapon  system. 
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The  cost-plus  environment  requires  a  whole  different  strat¬ 
egy  than  what  we've  used  in  the  past.  The  only  solution  in 
this  cost-plus  environment  is  teaming.  It  is  the  only  way  we 
are  going  to  find  out  the  information  fast  enough.  And  it's 
the  only  way  in  which  we  will  be  able  to  execute  the  program 
with  any  reasonable  cost  and  schedule  control. 

General  James  A.  Fain,  Jr. 


This  chapter  outlines  how  the  F-22  program  put  the  principles 
and  methods  described  in  chapters  2  and  3  into  practice.  The 
Government’s  Front  Office  Group  (FOG)  managed  the  program 
under  certain  constraints,  both  external  and  self-generated,  lb 
understand  how  the  program  applied  the  principles  and  methods, 
one  must  know  what  the  environment  was  and  how  it  drove  the 
program  structure,  how  the  program  office  planned  to  implement  the 
program,  and,  finally,  how  the  leadership  attempted  to  set  the  mores 
of  the  program — the  norms  for  the  F-22  team,  their  view  of  product 
orientation,  and  their  definition  of  team  member  roles. 

The  Environment 

The  ATF  demonstration/validation  phase  began  in  an  environ¬ 
ment  unlike  anything  that  anyone  connected  to  the  program  could 
remember.  After  the  peak  budgets  of  the  Reagan  administration. 
Department  of  Defense  funding  started  to  decline.  As  noted  in  chapter 
1,  acquisition  reform  dominated  1986.  The  Air  Force  initial  Advanced 
Tactical  Fighter  (ATF)  acquisition  strategy  called  for  a  competition 
among  three  or  four  weapon  system  contractors  and  two  engine  con¬ 
tractors  for  demonstration/validation.  In  the  next  phase,  engineering 
and  manufacturing  development  (EMD),  the  Air  Force  would  select 
one  weapon  system  and  one  engine  contractor  to  develop  the  ATF. 
Because  of  reduced  funding,  the  Air  Force  encouraged  the  seven 
weapon  system  contractors  that  had  competed  in  the  original  concept 
development  investigation  phase  to  form  teams.  In  this  way,  the  Air 
Force  could  maintain  competition  and  maximize  indiistry  involvement 
but  award  only  two  weapon  system  demonstration/validation  con¬ 
tracts  (at  a  cost  lower  than  that  of  the  original  three  or  four  contracts). 
(See  appendix  A  for  a  more  thorough  discussion  of  the  history  of  the 
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ATF  program.)  The  two  teams  competing  for  weapon  system  demon¬ 
stration/validation  were  the  Lockheed  Aircraft  Systems  Company- 
Greneral  Dynamics  Corporation-Boeing  Aircraft  Company  F-22  team 
and  the  Northrop  Aircraft  Division-McDonnell  Douglas  Corporation 
F-23  team.  Both  Pratt  &  Whitney  Government  Engine  Business 
(F119)  and  General  Electric  Aircraft  Engine  (F120)  received  contracts 
for  the  engine  demonstration/validation. 

For  the  demonstration/validation  phase,  the  Air  Force  chose  a 
fixed-price  contract  for  each  contractor  because,  in  a  competitive 
environment,  a  cost-type  contract,  which  pays  all  of  the  contractor’s 
costs,  would  encourage  the  contractors  to  spend  ever-higher  amounts 
to  help  secure  more  data  and  thus  increase  their  chances  of  winning 
the  development  and  then  production  contracts.  Following  the  exten¬ 
sive  risk  reduction  in  demonstration/validation,  the  program  office 
originally  planned  to  award  a  fixed-price  incentive  fee  contract  for 
EMD.  In  1988,  the  Department  of  Defense  required  all  develop¬ 
mental  contracts  to  be  cost  reimbursable,  forcing  a  major  change  in 
the  ATF  acquisition  strategy  for  the  EMD  phase. 

A  cost-type  contract  dramatically  changed  the  relative  roles  of 
Government  and  contractor  because  it  shifted  the  balance  of  risk.  In 
a  fixed-price  contract,  the  contractor  promises  to  deliver  a  product 
as  defined  in  the  contract,  for  which  it  is  paid  a  certain  amount  of 
money,  regardless  of  what  the  product  costs  the  contractor  to  produce. 
The  contractor  pays  for  any  cost  or  schedule  overrun,  lb  succeed,  a 
contractor  needs  well-defined  requirements,  usually  described  in 
detailed  specifications.  As  happened  in  the  Air  Force  C-17  cargo 
aircraft  and  Navy  A-12  attack  aircraft  programs,  a  fixed-price  devel¬ 
opment  contract  has  great  potential  to  create  an  adversarial  relation¬ 
ship  between  the  contractor  and  the  Gk)vemment. 

In  a  cost-type  contract,  the  contractor  promises  its  best  effort  in 
delivering  a  product  for  which  the  Government  pays  all  allowable 
costs,  including  any  overrim.  The  Government  assumes  all  cost  risk 
and  responsibility.  When  the  Gk)vernment  cannot  tightly  define  the 
specifications  and  requires  design  trade-offs,  cost-type  contracts  are 
best.  One  advantage  of  a  cost-type  over  a  fixed-price  contract  is  that  it 
reduces  the  potential  for  adversarial  relationships  and  increases  the 
potential  to  build  a  successful  industry/Government  team.  A  disad¬ 
vantage  is  that  the  contractor,  unless  motivated  in  some  other  way 
(such  as  by  an  award  fee),  may  be  less  inclined  to  tightly  control  costs. 

Because  the  Government  incurred  the  cost  risk  and  did  not  want 
to  see  a  cost  overrun,  the  program  leaders  wanted  the  Government 
to  take  part  in  all  decisions.  The  program  office  also  realized  that  it 
needed  to  continue  to  balance  cost,  schedule,  and  performance  with 
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a  structured,  disciplined  management  approach  in  which  the 
Government  gave  clear  guidance.  To  be  successful,  the  program 
office  needed  to  rely  on  contractor  and  government  working  togeth¬ 
er  to  deliver  the  fighter  that  the  operational  user  expected. 

Implementation:  Multifunctional  Teams 

The  F-22  program  office  needed  to  establish  a  way  to  allow  the 
Government  to  participate  but  not  to  be  a  roadblock.  Information 
had  to  flow  quickly  within  the  Govemment/contractor  team  to  allow 
time-sensitive  decisions  to  be  made.  The  only  way  to  accomplish 
these  objectives  was  to  depend  on  the  working-level  managers  to 
provide  data  as  well  as  make  decisions. 

The  F-22  management  team  realized  that  the  success  of  the  F-22 
rested  on  lower-tier  members  of  the  Government/contractor  team. 
For  them  to  succeed,  the  program  leadership  needed  to  empower 
them  to  make  decisions  in  their  areas  while  keeping  higher  levels 
informed.  Each  team  had  to  understand  their  roles,  their  decision¬ 
making  authority,  and  they  had  to  understand  when  to  elevate 
issues  to  higher  level  decisionmakers. 

To  define  a  team’s  decision  space,  the  program  used  the  concept  of 
fenceposts.  The  F-22  FOG  believed  in  empowering  all  members  of  the 
team  to  work  on  issues  and  make  decisions  appropriate  to  their  posi¬ 
tion  and  individual  capability.  For  example,  the  leader  of  the  cockpit 
team  should  worry  about  the  weight  and  the  cost  of  the  canopy  and 
would  be  responsible  for  making  the  necessary  trade-offs  to  meet  the 
users’  needs  for  a  canopy  and  the  team  requirement  to  meet  cost  and 
schedule  performance  requirements.  Fain  established  what  decisions 
each  of  his  team  leaders  should  make  and  what  decisions  they  should 
pass  up  to  higher  levels.  Fain,  James  (Micky)  Blackwell  (Lockheed’s 
F-22  program  manager),  and  Walter  Bylciw  (Pratt  &  Whitneys  F-119 
program  manager)  approved  a  budget  for  each  team,  approved  an  allo¬ 
cation  for  weight  (a  critical  parameter  for  an  aircraft),  and  suballo¬ 
cated  technical  requirements  when  needed.  They  also  approved  the 
plan  that  each  team  was  to  follow.  Thus  each  team  had  its  own  cost, 
schedule,  and  performance  targets.  In  effect,  each  team  was  its  own 
program  developing  its  particular  product,  and  each  team  leader 
served  as  that  product’s  program  director. 

Fain,  Blackwell,  and  Bylciw  recognized  that  this  empowerment 
would  be  a  new  experience  for  the  lower  tier  workers.  Appl3ring 
concurrent  leadership  called  for  multifunctional  teams,  whereas  the 
working-level  personnel  had  traditionally  focused  on  a  specific  exper¬ 
tise  (such  as  electrical  engineering,  mechanical  engineering,  mainte¬ 
nance,  or  finance).  They  had  never  been  trained  to  balance  the  design 
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of  a  complete  product;  typically  they  had  concentrated  on  perfecting 
their  piece  of  the  product.  Under  the  new  approach  (discussed  in 
more  detail  in  chapter  7),  the  team  members  had  to  look  at  each  prob¬ 
lem  from  a  higher  level  perspective,  incorporating  the  functional  con¬ 
cerns  of  other  team  members  with  those  based  on  their  own  area  of 
expertise.  This  new,  broader  perspective  required  training,  much  of 
which  came  from  interaction  with  the  other  functional  members  of 
the  teams. 

To  support  the  new  team  approach,  the  program  needed  a  new  set 
of  integrated  tools  that  would  provide  for  a  disciplined,  well-planned, 
easily  tracked  program  and  allow  the  team  members  and  program  lead¬ 
ers  to  measure  progress  and  success.  Chapter  8  discusses  these  tools. 

The  Team  Norms 

Fain  believed  his  role  included  establishing  the  norms,  or  mores, 
of  the  ATF/F-22  team  for  government  as  well  as  contractor  personnel. 
At  the  start  of  the  EMD  phase,  Fain  traveled  to  each  contractor  and 
most  subcontractors  to  personally  explain  his  views  on  the  program 
and  the  standards  to  which  he  would  hold  each  team  member.  The 
bedrock  of  all  of  his  thoughts,  what  he  called  his  team  philosophy, 
derived  from  his  three  personal  pillars:  “I  am  going  to  harp  on  three 
things:  Integrity,  Teamwork,  and  Logic,” 

Integrity 

To  Fain,  integrity  was  clearly  the  most  important  tenet  (and  the 
reason  it  is  the  first  principle  of  the  20  listed  in  Figure  2-1).  Integrity 
in  the  F-22  program  demanded  that  every  team  member  conduct  all 
business  relationships  honestly  and  ensure  that  the  receiver  of  a 
message  understands  it.  Fain  made  it  very  cle£Lr  to  all  F-22  person¬ 
nel,  including  those  in  his  program  office,  at  other  Government  loca¬ 
tions,  and  at  the  contractors^  or  subcontractors’  facilities,  that  he 
expected  complete  integrity  from  everyone  involved  in  the  F-22 
development  program. 

Teamwork 

Fain  recognized  that  it  was  the  combined  efforts  of  thousands  of 
individuals  that  made  the  demonstration/validation  phase  success¬ 
ful  and  that  the  engineering  and  manufacturing  phase  required  the 
three  weapon  system  team  members  (Lockheed,  Boeing,  and 
General  Dynamics)  to  continue  to  work  well  together  and  with 
the  engine  contractor  (Pratt  &  Whitney).  As  the  new  phase  of  the 
program  started,  each  contractor  would  bring  more  people  into  the 
program.  Fain  did  not  want  the  successes  of  demonstration/valida- 
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tion  and  its  hard-earned  teamwork  to  fall  away  with  the  new  work¬ 
ers.  He  emphasized  three  items. 

First,  he  encouraged  the  continuation  of  open,  timely,  and  effec¬ 
tive  communication  between  and  among  the  contractors  and  all 
Government  team  members.  Because  the  contractor  organization 
divided  major  responsibilities  among  three  large  contractors  span¬ 
ning  the  country  (Lockheed  in  Marietta,  GA;  Boeing  in  Seattle,  WA; 
and  General  Dynamics  in  Fort  Worth,  TX),  he  knew  that  open  and 
continuous  communication  was  vital  to  successfully  develop  the 
weapon  system. 

Second,  in  keeping  with  his  belief  that  a  problem  identified  early 
was  often  a  problem  easily  and  cheaply  solved,  Fain  directed  each 
team  to  identify  obstacles  to  its  program  quickly  and  then  to  report 
them  as  soon  as  the  team  members  decided  they  could  not  overcome 
the  obstacles  themselves. 

Third,  Fain  wanted  all  individuals  to  maintain  close  working 
relationships  with  their  counterparts  on  other  teams  and  be  respon¬ 
sive  to  the  users’  needs.  For  program  office  personnel,  this  meant 
they  should  stay  in  close  contact  with  their  team  members  at  Air 
Combat  Command,  the  users.  The  program  office  needed  to  be 
responsive  to  user  needs  (which  included  the  individuals  who  repair 
and  maintain  the  system).  Contractor  personnel  were  to  work  close¬ 
ly  with  both  program  office  team  members  (a  contractor’s  tradition¬ 
al  customer)  and  operational  team  members. 

With  this  focus  on  teamwork,  Fain  attempted  to  smash  an  age- 
old  paradigm.  In  the  past,  the  contractor  had  designed  a  weapon 
system  for  the  government  inside  the  company  and,  as  it  went  along, 
presented  results  to  the  government  (usually  just  the  program 
office).  Fain  understood  the  immense  complexity  of  developing  the 
F-22,  especially  with  responsibility  spread  among  four  major 
contractors  (including  the  engine  contractor).  He  had  to  get  the 
contractor  team  members  to  change  the  traditional  way  of  doing 
their  work  and  develop  a  team  approach  that  included  both  the 
government  program  office  and  the  user. 

Logic 

Fain’s  third  pillar  for  the  F-22  team  was  logic,  or  what  he  frequent¬ 
ly  called  common  sense.  He  shared  his  belief  that  rules  and  regulations 
were  merely  guidelines.  In  his  view,  the  only  reason  regulations  had 
been  written  was  to  try  to  correct  some  problem  experienced  in  the  past. 
Therefore,  he  wanted  everyone  on  the  F-22  program  to  let  common 
sense,  not  rules  and  regulations,  drive  all  their  decisions. 
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Product  Orientation 

To  ensure  ownership  (Principle  6),  the  founding  fathers  focused 
the  teams  on  tangible  products,  ideally  a  physical  entity  such  as  a 
piece  of  hardware  (the  software  component  as  well  as  the  physical 
hardware),  like  a  radar,  an  engine  turbine,  a  landing  gear,  a  cockpit, 
or  an  avionics  suite.  To  keep  focused  on  the  final  objective,  the 
weapon  system,  each  team  needed  to  understand  how  its  subproduct 
related  to  the  overall  product  (brakes,  landing  gear,  airframe,  and 
completed  jet),  what  their  product  was  to  do,  how  it  would  be  built, 
how  it  would  be  supported,  and  how  the  individuals  using  and  main¬ 
taining  their  product  would  be  trained. 

The  Roles  of  Organizations  as  Team  Members 

An  important  aspect  of  running  a  disciplined  program  is  estab¬ 
lishing  clear  roles  and  responsibilities  for  the  members  of  a  team  as 
well  as  for  the  organizations  that  make  up  a  team.  The  major  orga¬ 
nizations  making  up  the  weapon  development  team  are  the  users, 
the  program  office,  and  the  contractor. 

The  Users 

The  most  important  member  of  the  development  team  is  the  user. 
The  system  exists  to  fulfill  the  users'  need,  and  users  define  the 
requirements.  To  help  ensure  the  best  product,  the  users  need  to  state 
their  requirements  in  a  functional  manner.  General  George  S.  Patton, 
Jr.  advised,  “Never  tell  people  how  to  do  things.  Tell  them  what  to  do 
and  they  will  surprise  you  with  their  ingenuity."'  This  holds  true  in 
acquisition.  Users  should  state  their  requirements  without  specifying 
the  solution.  As  part  of  this  process,  the  users,  with  the  help  of  intel¬ 
ligence  specialists,  such  as  the  service  intelligence  experts  and  the 
Defense  Intelligence  Agency,  define  potential  threats,  determine  like¬ 
ly  scenarios  in  which  they  will  use  the  new  weapon  system,  describe 
where  and  how  they  will  use  it  (which  defines  the  operating  environ¬ 
ment),  and  define  how  they  intend  to  maintain  the  equipment  and 
what  level  of  skilled  technicians  will  repair  it. 

The  users  also  need  to  work  closely  with  the  program  office  to 
understand  available  options  to  fulfill  their  requirements,  to  under¬ 
stand  the  affect  of  their  requirements  on  the  cost  and  complexity  of 
the  weapon  system,  and  to  be  ready  to  adjust  requirements  to  balance 
operational  capability  against  weapon  cost  and  complexity.  The  officer 
responsible  for  ATF  requirements  at  Tactical  Air  Command  during  the 
demonstration/validation  phase,  Major  General  (then  Colonel)  David 
J.  McCloud,  described  a  good  example  of  such  a  trade-off: 
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It  is  easy  to  say,  ‘We  need  a  9G  aircraft”  [able  to  turn  at  a 
speed  that  results  in  a  pull  on  the  plane  of  nine  times  the 
force  of  gravity].  The  question  gets  much  more  complex 
when  the  program  office  asks,  “Do  you  want  9Gs  immedi¬ 
ately  after  takeoff,  or  will  you  settle  for  7Gs  at  takeoff  and 
9Gs  when  you  get  to  altitude?”  It  turns  out  that  the  differ¬ 
ence  is  500  or  600  pounds  of  aircraft  structural  weight  and 
a  lot  of  engineering  time  that  goes  into  beefing  up  the  air¬ 
craft  structure  to  give  you  the  ability  to  pull  9Gs  right  after 
takeoff.  The  point  is,  we  don’t  need  9Gs  right  at  takeoff. 

Where  we  need  QGs  is  in  the  fight  arena. 

The  Program  Office 

The  system  program  office  has  the  responsibility  to  make  sure  the 
users  get  what  they  need.  Its  role  is  to  form  the  acquisition  team, 
establish  the  team  environment,  and  work  with  the  users  as  they 
identify  their  needs  and  define  their  requirements.  The  program 
office  must  then  translate  those  reqxiirements  into  terms  and  a  struc¬ 
ture  that  are  meaningful  to  the  contractor.  In  so  doing,  they  must 
make  sure  the  contractor  fully  understands  what  the  users  need. 

The  program  office  also  needs  to  work  with  the  contractor  to  devel¬ 
op  and  provide  options  to  the  users  and  to  explain  to  them  how  long 
each  option  will  take  to  deliver  and  how  much  each  will  cost.  It  should 
also  provide  an  *^80  percent”  option  that  meets  the  most  important 
user  requirements  but  that  can  be  developed  in  less  time,  for  less 
money,  or  both.  With  a  full  set  of  options  the  users  can  determine  the 
cost  of  meeting  their  requirements  and  decide  which  option  is  best, 
given  the  available  government  funding.  The  program  office  must  also 
ensure  that  the  contractor  has  laid  out  a  sound  program  with  an 
appropriate  level  of  risk. 

Finally,  as  the  prime  agent  responsible  for  delivering  the  equip¬ 
ment,  the  most  important  role  of  the  program  office  is  to  set  and 
meet  everyone's  expectations.  For  the  user  to  understand  what  is 
really  possible,  how  much  time  the  development  will  take,  and  how 
much  it  will  cost,  the  program  office  must  develop  viable  options 
with  viable  cost  and  schedule  projections.  For  contractors  to  fully 
understand  what  users  need,  and  to  operate  under  a  fair  contract 
that  encourages  them  to  develop  it,  the  program  office  must  appro¬ 
priately  set  contractor  expectations  about  the  work  required  and  the 
method  of  contracting. 

The  Contractors 

The  contractors  have  the  hardest  job:  to  understand  the  user's 
requirements  and  generate  the  options  to  fulfill  them.  They  must 
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develop  the  ideas  and  find,  or  develop,  the  necessary  technology  to 
transform  them  into  an  actual  weapon. 

The  contractor  lays  out  a  program  to  develop  a  system  that  will 
meet  the  operational  requirements  with  the  appropriate  amount  of 
risk,  conducts  the  program,  and  produces  the  actual  weapon  system. 

Figure  4-1  illustrates  the  roles  and  responsibilities  of  the  operator, 
the  program  office,  and  the  contractor  on  the  weapon  system  develop¬ 
ment  team.  The  circles  represent  the  activity  of  each  team  member. 
The  significant  overlap  of  the  user  and  program  office  circles  repre¬ 
sents  the  high  amount  of  interaction  that  should  take  place  between 
the  two  groups.  All  three  circles  overlap  to  a  great  extent,  indicating 
that  much  of  the  activity  requires  the  participation  of  all  three  team¬ 
mates.  The  contractor  and  user  circles  do  not  have  much  overlap 
outside  the  program  office  circle.  Program  managers  generally  agree 
that  program  office  attendance  at  discussions  between  the  user  and 
the  contractor  can  help  ensure  that  the  Government  (as  the  user)  does 
not  provide  contractual  direction  to  the  contractor  and  that  the 
contractor  fully  explains  its  information  to  the  user. 


Figure  4-1.  Weapon  System  Development  Team  Member  Roles 


Concepts/technologies/product 


Photo  Randy  O'BrKzar,  of  Bering  Aifcmft- 

l^rication  of  tti@  F-22  began  December  8, 1993  as  a  milling  machine  operator  used  a  five-axis  milling  machine  to  cut  a  blodc  of 
titanium  to  form  a  forward  boom  keelson  panel. 
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Fundamental  to  the  ultimate  success  of  a  new  program  is  an 
informed  trade-off  between  user  requirements,  on  the  one 
hand,  and  schedule  and  cost,  on  the  other. 

President's  Blue  Ribbon  Commission  on  Defense  Management, 
A  Quest  for  Excellence:  Final  Report  to  the  President 


As  described  in  chapter  3,  the  F-22  program  leadership  viewed 
managing  risk  as  the  key  to  program  success.  As  depicted  in 
Figure  5-1,  managing  risk  required  defining  requirements, 
using  proven  technology,  and  using  a  management  system  that 
supported  a  disciphned  development  program.  This  and  the  next  three 
chapters  show  how  the  F-22  program  implemented  these  ideas. 

The  Packard  Commission  View 

President  Ronald  Reagan’s  Commission  on  Defense  Management, 
led  by  David  Packard,  identified  establishing  weapon  system  require¬ 
ments  as  a  fundamental  acquisition  problem.  The  Commission  report¬ 
ed,  “Problems  with  the  present  defense  acqiiisition  system  begin  with 
the  establishment  of  approved  ‘military  requirements’  for  a  new 
weapon,  a  step  that  occxurs  before  development  starts.”  The  report  stat¬ 
ed  that  the  services  typically  define  requirements  in  one  of  two  ways, 
user  pull  or  technology  push.  In  user  pull  a  user  identifies  the  weak¬ 
nesses  of  current  systems  against  projected  threats  and  generates  the 
characteristics  of  future  weapons — without  fully  understanding  the 
likely  affect  of  these  requirements  on  cost,  schediile,  or  maintenance. 


Figure  5-1.  Elements  of  Risk  Management:  Requirements 
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In  technology  push  the  research  community  develops  a  new  technolo¬ 
gy  and  tries  to  convince  users  to  make  use  of  it.  The  downsides  to  tech¬ 
nology  push  are  that  the  technology  may  not  fully  meet  the  user’s 
operational  needs,  may  not  be  fully  developed,  or  may  lead  to  other 
problems  (e.g.,  the  jet  engine  greatly  improved  aircraft  speed  but  was 
susceptible  to  damage  from  stones  and  other  loose  objects  while  the 
aircraft  was  on  the  ground  with  the  engines  running). 

The  Packard  Commission  correctly  saw  that  the  answer  to  the 
longstanding  requirements  problem  was  to  balance  performance  and 
cost.  However,  the  Commission’s  method  of  doing  so  violated  the 
Advanced  Tactical  Fighter  (ATF)  principles  of  involving  everyone  and 
ensuring  ownership.  The  final  report  stated: 

Generally,  users  do  not  have  sufficient  technical  knowledge 
and  program  experience,  and  acquisition  teams  do  not 
have  sufficient  experience  with  or  insight  into  operational 
problems,  to  strike  this  critical  balance  [between  perfor¬ 
mance  and  cost].  It  requires  a  blend  of  diverse  backgrounds 
and  perspectives  that,  because  the  pressures  of  goldplating 
can  be  so  great,  must  be  achieved  at  a  very  high  level  in 
DOD....  We  recommend,  therefore,  that  the  JRMB  [Joint 
Requirements  Management  Board]  be  restructured  to 
make  such  trade-offs....  It  should  evaluate  major  trade-offs 
as  a  program  progresses. 

The  ATF  Approach 

The  ATF  method  of  balancing  cost  and  performance  relied  on  the 
stakeholders  cooperatively  defining  and  refining  both  the  require¬ 
ments  and  the  proposed  solution.  The  ATF  founders  labored  over  long¬ 
standing  problems  with  requirements.  As  ATF  Technical  Director,  Eric 
E.  Abell,  said:  “Our  objective  was  to  balance  requirements  against 
proven  capability.  Only  then  could  you  ensure  that  you  really  could  do 
what  the  operator  wanted  and  thus  meet  their  expectations.”  Instead 
of  rel3dng  on  a  distant  headquarters  group  (the  Packard  solution)  to 
conduct  the  necessary  trade-offs  and  establish  system  requirements, 
the  leaders  formed  an  integrated  team  of  operators  from  TAG,  acqui¬ 
sition  experts  from  the  program  office,  and  the  competing  Lockheed 
YF-22  and  Northrop  YF-23  aircraft  teams  and  the  competing  Pratt  & 
Whitney  F119  and  General  Electric  F120  engine  teams.  The  ATF  pro¬ 
gram  system  for  defining  requirements  is  shown  in  Figure  5-2. 

The  process  started  in  1986,  following  the  award  of  the  contracts 
for  the  demonstration/validation  phase  of  the  ATF  program.  Based 
upon  the  preliminary  ATF  system  operational  concept  provided  by  the 
TAG,  the  program  office  drafted  an  initial  list  of  the  goals  for  the 
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Bottom  Line:  The  process  results  in  match  of  requirements  and  doable  technologies  to 
form  the  basis  of  an  executable  program  at  the  start  of  the  engineering  and  manufactur¬ 
ing  development  phase. 
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weapon  system.  The  program  office  then  worked  closely  with  TAG  to 
refine  the  list  and  form  the  system  requirements  document.  The  com¬ 
peting  contractors  used  this  document  as  a  basis  for  their  weapon  sys¬ 
tem  design  (the  jet  as  well  as  the  support  and  training  system),  which 
would  evolve  and  become  known  as  their  preferred  system  concept. 

In  a  series  of  System  Requirements  Reviews,  each  contractor 
demonstrated  how  it  would  meet  the  Government’s  goals  for  the  ATF. 
Initially  the  contractors  felt  that  they  could  meet  the  diverse  and 
demanding  requirements  established  in  the  system  requirements  docu¬ 
ment.  Abell  made  it  clear  that,  before  the  specifications  were  finalized 
for  the  next  development  phase,  engineering  and  manufacturing  devel¬ 
opment  (EMD),  that  the  contractors  must  prove — ^by  demonstration  and 
analysis — ^that  they  could  meet  the  requirements.  As  the  program  pro¬ 
gressed  each  contractor  discovered  it  could  not  meet  all  of  the  goals. 
During  individual  System  Specification  Technical  Interchange 
Meetings,  the  contractors  presented  their  engineering  results  and  dis¬ 
cussed  with  the  Government  program  office/operator  team  how  they 
planned  to  update  their  preferred  system  concept. 

Following  the  system  specification  meetings,  the  program  office 
and  the  users  updated  the  preliminary  ATF  system  specifications. 
These  sessions  exposed  the  TAG  users  to  the  pros  and  cons  of  each 
technology  and  the  preferred  contractor  solutions.  The  users  saw  the 
projected  cost,  reliability,  and  complexity  of  each  concept — thus  fixing 
what  the  Packard  Gommission  identified  as  a  key  failing  of  the  acqui¬ 
sition  requirements  process.  Also,  through  demonstrations  and  tests, 
the  contractor  proved  the  validity  of  the  technology  it  planned  to 
employ  (answering  the  technology  push  concerns). 

The  preliminary  system  specifications  became  the  functional  spec¬ 
ifications  used  in  the  Government’s  request  for  proposal  (RFP)  for  the 
EMD  phase.  Each  contractor’s  preferred  system  concept  design 
became  their  aircraft  design  proposal.  This  iterative  process  of  adjust¬ 
ing  requirements  based  on  proven  technologies  met  the  needs  of  the 
users,  the  program  office  personnel,  and  the  contractors.  All  three 
groups  set,  and  met,  appropriate  expectations  and  fully  imderstood 
the  reasons  behind  the  requirements  and  the  preferred  solutions, 
because  all  three  groups  had  helped  build  them. 

The  Summit  Process 

Based  on  negative  experience  with  earlier  programs  (such  as  the 
B-1  bomber),  Air  Force  Ghief  of  Staff,  General  Larry  D.  Welch,  wanted 
to  ensure  that  the  ATF  requirements  were  updated  on  a  regular  basis. 
He  required  the  Program  Director  to  present  the  ATF  requirements  to 
the  Gommander  of  Tactical  Air  Gommand  and  himself  annually  so  that 
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the  operational  requirements  document  remained  current.  He  hoped  to 
preclude  problems  when  the  program  came  up  for  operational  testing 
(such  as  having  to  test  to  requirements  no  longer  needed  by  the  users 
that  the  weapon  system  might  not  be  able  to  meet).  Welch  also  wanted 
the  Program  Director  to  understand  what  the  user  expected  from  the 
weapon  system  and  find  out,  from  the  program  office  standpoint,  if 
there  were  any  problems  in  meeting  those  validated  needs. 

Toward  the  end  of  the  demonstration/validation  phase,  after  the 
users  on  the  ATF  team  had  approved  the  preliminary  system  speci¬ 
fications,  the  Program  Director  met  with  the  Commander  of  Tactical 
Air  Command  and  then  the  Air  Force  Chief  of  Staff,  who  formally 
approved  the  requirements.  Both  the  ATF  program  office  and  the 
warfighting  command  found  this  process,  later  known  as  the 
Summit  Process,  immensely  helpful  in  solidifying  requirements  and 
preventing  unnecessary  changes  by  members  of  the  acquisition 
process  outside  of  the  TAC  users. 

Air  Combat  Command,  in  close  coordination  with  the  program 
office,  continues  to  refine  and  update  the  requirements  by  comparing 
anticipated  performance  with  operational  needs.  If  there  is  a  project¬ 
ed  shortfall  in  fighter  performance,  Air  Combat  Command  and  the 
program  office  identify  solutions  to  meet  the  user's  needs.  The  results 
of  these  requirement  reviews  and  updates  continue  to  be  briefed 
annually  as  part  of  the  F-22’s  Summit  Process. 

Summary 

Allowing  the  user,  program  office,  and  contractors  to  work  close¬ 
ly  together  in  stating  requirements  and  demonstrating  the  ability  to 
meet  them  greatly  improves  the  odds  of  developing  a  weapon  that 
can  carry  out  warfighter  missions.  An  important  benefit  is  that  the 
users  see  the  impact  of  their  requirements  on  the  cost  and  complex¬ 
ity  of  the  weapon  system  and  can  then  explore  the  possibility  of 
refining  their  requirements  to  lower  either  element. 

Major  General  David  McCloud,  the  former  chief  of  ATF  require¬ 
ments  at  TAC  during  the  demonstration/validation  phase,  summed  up 
the  requirements  refinement  process: 

The  general  consensus  of  those  involved  is  that  the  ATF 
program  has  been  very  successful  especially  in  the  very 
early  part  of  the  program  when  we  were  heavily  involved 
in  the  requirements  trade-offs.,..  The  one  thing  that  made 
the  ATF  program  such  a  success  was  small  organizations, 
tightly  tied  together,  but  willing  to  challenge  each  other 
and  listen  and  argue  and  debate  and  come  up  with  a 
reasonable  answer. 
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We've  demonstrated  all  the  technologies  to  be  employed.  There 
are  no  miracles  required  here.  There  are  no  technologies  we 
don't  have  a  handle  on. 


Colonel  Wallace  T.  Bucher 
at  the  start  of  the  engineering  and 
manufacturing  development  phase 


After  defining  the  requirements  of  the  weapon  system,  the 
second  element  of  the  Advanced  Tactical  Fighter  (ATF) 
program  approach  to  risk  management  was  to  develop  and 
prove  the  technology  required  for  the  design  and  production  of 
the  weapon  system  (see  Figure  6-1).  Recalling  the  13th  principle, 
“Have  what  you  need  for  the  effort,”  the  founders  mandated  that 
the  contractors  could  not  propose  a  solution  for  the  engineering 
and  manufacturing  development  (EMD)  phase  unless  they  had 
sufficiently  proved  the  technology  during  the  demonstration/valida¬ 
tion  phase. 

The  initial  strategy  for  the  ATF  team  demonstration/validation 
phase  called  for  contracting  with  three  or  four  of  the  seven  airframe 
prime  contractors  who  had  participated  in  the  initial  concept  develop¬ 
ment  investigation  phase.  Tight  funding  required  the  contractors  to 
apply  the  limited  money  to  the  areas  with  the  highest  risk.  The  plan 
called  for  the  competing  contractors  to  conduct  Critical  Technology 
Demonstrations  aimed  at  the  highest  risk  area,  primarily  avionics. 
The  contractors  were  to  use  computer  models  to  assess  the  aerody¬ 
namic  performance  of  their  proposed  aircraft. 


Figure  6-1.  Elements  of  Risk  Management:  Technology 
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The  Packard  Commission  Recommendation 

In  April  1986,  in  the  midst  of  the  demonstration/validation  source 
selection,  the  program  office  plans  suddenly,  and  historically,  changed. 
The  Acquisition  Task  Force  of  President  Ronald  Reagan’s  Blue  Ribbon 
Commission  on  Defense  Management  submitted  an  interim  report 
on  improving  the  national  acquisition  process,  A  Formula  for  Action, 
One  of  the  recommendations  was  to  increase  the  use  of  prototypes  in 
the  early  stages  of  development: 

We  recommend  a  high  priority  on  building  and  testing 
prototype  systems  to  demonstrate  that  new  technology 
can  substantially  improve  military  capability,  and  to  pro¬ 
vide  a  basis  for  realistic  cost  estimates  prior  to  a  full-scale 
development  decision. 

The  Air  Force  responded  to  the  Commission’s  report  by  amending 
its  guidance  in  the  ATF  Program  Management  Directive,  This  new 
guidance  directed  the  ATF  program  office  to  include  the  development, 
fabrication,  and  test  of  two  prototype  air  vehicles  in  the  demonstra¬ 
tion/validation  program.  Each  contractor  was  to  “fabricate  and  demon¬ 
strate  a  grotmd-based  prototype  Avionics  Integration  Labora-tory,  and 
conduct  active  sensor  testing  aboard  an  Avionics  Flying  Laboratory,” 
(See  appendix  C  for  a  discussion  of  the  use  of  prototypes.) 

Flying  prototypes  are  in  the  short  term  more  expensive  than 
paper  studies.  To  follow  Air  Force  guidance  and  keep  the  program 
within  the  approved  funding,  the  ATF  program  office  took  advantage 
of  the  information  it  had  seen  in  the  initial  program  phase  by  encour¬ 
aging  contractors  to  form  two  competing  teams. 

In  October  1986,  the  Air  Force  awarded  two  ATF  contracts,  the 
YF-22  to  Lockheed  teamed  with  Boeing  and  General  Dynamics,  and 
the  YF-23  to  the  team  of  Northrop  and  McDonnell  Douglas.  These 
54-month,  firm  fixed-price  contracts  required  their  best  effort  in  the 
initial  design  and  testing  of  the  ATF.  Each  contractor  was  to  fly  two 
prototype  air  vehicles,  develop  a  ground  avionics  prototype  (with  an 
avionics  flying  laboratory),  and  develop  initial  system  specifications. 

The  Air  Force  awarded  General  Electric  and  Pratt  &  Whitney 
competing  contracts  to  develop  prototype  engines  under  the  ATF 
Engine  program.  Support  of  the  prototype  air  vehicle  required  each 
contractor  to  deliver  six  flight-qualified  ATF-protot3q)e  engines  (two 
for  each  of  the  YF-22s,  two  for  each  of  the  YF-23s,  and  two  spares)  plus 
a  number  of  groimd  demonstrator  engines  (as  originally  planned  under 
the  previous  Joint  Advanced  Fighter  Engine,  or  JAPE,  program). 


Pho^o  courtesy  of  Pratt  It  Whitney, 


An  F1 19  engine  prototype  undergoes  pre-flight  tests. 


58  Acquisition  for  the  21st  Century 


The  DemonstrationA^alidation  Technology  Program 

Based  on  the  initial  airframe  concepts  and  the  technology  in 
development  at  various  Air  Force  laboratories,  the  ATF  program 
office  main  technical  concerns  involved  three  components: 

1.  The  aerodynamic  performance  of  the  fighter  in  a  low-observ¬ 
able  (stealth)  platform 

2.  The  advanced  super  cruise  engine 

3.  The  integrated  avionics  in  the  aircraft. 

Aerodynamic  Performance  in  a  Low-observable  Platform 

Many  Air  Force  and  the  Department  of  Defense  leaders  had  seri¬ 
ous  doubts  as  to  whether  a  stealth  aircraft  could  be  made  maneuver- 
able  (in  1986  few  people  were  cleared  into  stealth  aircraft  programs; 
those  that  were  knew  of  the  F-117  and  the  B-2 — neither  of  which 
approached  the  maneuverability  of  an  F-15).  Many  feared  that  the 
stealth  constraints  (such  as  leading-  and  trailing-edge  alignment) 
would  severely  limit  high-speed,  "turn-and-bum"  fighter  performance. 

Advanced  Supercruise  Engine 

The  capabilities  of  the  new  engines  emerging  from  the  JAFE  (later 
renamed  the  ATF  Engine)  program  would  allow  a  fighter  to  cruise 
faster  than  the  speed  of  sound  for  sustained  periods.  Until  then,  such 
speed  required  the  use  of  fuel-consuming  afterburners.  Many  in 
Washington  and  the  aerospace  community  doubted  that  supercruise 
was  possible,  even  with  new  technology. 

Integrated  Avionics 

Technological  improvements  were  rapid  in  the  avionics  (aircraft 
electronics)  industry.  The  latest  fighter,  the  Navy  F/A-18,  had  the 
most  electronically  advanced  cockpit  and  the  most  difficult  and 

Figure  6-2.  Elements  of  the  ATF  Demonstration/Validation  Phase 
— - 

•  Prototype  air  vehicles 

•  Radar  cross-section  models 

•  Test  engines 

•  Avionics  prototypes 

•  Reliability,  maintainability,  and  support 

•  Materials 
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demanding  set  of  controls  of  any  aircraft  flying.  The  ATF  avionics 
suite  had  to  be  fully  integrated  to  give  the  pilot  situational  awareness 
(an  easy-to-understeind  view  of  the  environment).  The  diversity  and 
complexity  of  avionics  and  the  poor  performance  of  past  avionics 
programs  led  the  program  office  and  contractors  to  view  avionics  as 
the  technological  area  of  highest  risk. 

Figure  6-2  shows  the  elements  of  the  ATF  demonstration/vahda- 
tion  phase.  This  phase  attempted  to  reduce  the  risk  in  the  three  areas 
of  technical  concern  by  developing  and  testing:  prototype  air  vehicles, 
radar  cross-section  models,  test  engines,  and  avionics  prototypes. 

In  addition,  the  ATF  program  needed  to  build  the  foundation  for 
key  technologies  that  would  be  used  in  the  EMD  phase.  In  particu¬ 
lar,  the  program  was  seeking  improvements  in  materials  and  in  reli¬ 
ability,  maintainability,  and  support. 

The  final  element  of  the  ATF  demonstration/validation  phase, 
which  included  a  multitude  of  tasks  from  creating  and  refining  opera¬ 
tional  requirements  to  detailing  the  ATF  design,  focused  on  modeling 
and  simulation. 

Prototype  Air  Vehicles 

The  purpose  of  building  prototype  air  vehicles  was  to  demonstrate 
the  aerodynamic  performance  of  the  basic  shape  of  the  low-observable 
aircraft.  Prime  contributors  to  an  aircrafts  radar  signature  are  its  pro¬ 
file  and  planform.  Both  the  (Government  and  the  contractors  wanted 
to  see  how  well  the  uniquely  shaped  ATF  would  fly.  The  test  aircraft 
would  also  corroborate  wind  tunnel  data  and  provide  data  with  which 
teams  could  calibrate  computer  models.  The  prototypes  had  standard, 
current-generation  flight  controls  and  avionics  and  did  not  employ 
low-observable  coverings  or  treatments.  Nor  did  the  prototype  aircraft 
incorporate  the  internal  subsystems  that  would  form  its  operational 
configuration  (which  would  be  finalized  with  the  other  operational 
design  elements  during  EMD). 

The  program  philosophy  for  prototype  air  vehicle  flight  testing  dif¬ 
fered  from  that  in  other  programs.  Traditionally,  the  Government 
owned  the  test  aircraft  and,  with  the  contractor,  would  determine 
what  to  test.  Typically,  competing  aircraft  would  fly  similar  profiles  to 
evaluate  the  performance  of  one  against  that  of  the  others.  In  the  ATF 
program,  however,  the  contractor  retained  ownership  of  the  air  vehi¬ 
cles  and  determined  what  aspects  to  demonstrate.  As  the  contractor’s 
engineering  and  manufacturing  program  manager,  Lockheed  Vice 
President  James  E.  “Micky”  Blackwell,  stated,  "The  sole  contractual 
requirement  for  the  prototype  was  that  it  had  to  take  off." 


Rioto  ccMir^y  erf  Nortltfyp-Giwrari. 

Leading  and  trailing  edge  alignments  are  shmivn  in  the  pianform  view  of  theYF-23.The  alignments  aid  in  reducing  the  aircraft’s 
ladar  signature. 


Managing  the  Technology  61 


The  four  aircraft  were  flight  tested  from  August  27  until 
December  28,  1990.  As  shown  in  Table  6-1,  the  Lockheed/Boeing/ 
General  Dynamics  YF-22  and  the  Northrop/McDonnell  Douglas  YF-23 
successfully  completed  124  sorties  and  logged  157.4  flying  hours. 
During  that  time  the  competing  contractors  demonstrated  handling 
qualities,  performance  at  altitudes  above  50,000  feet,  and  sustained 
supercruise  of  more  than  1.3  Mach.  Both  the  YF-22  and  Yr-23  demon¬ 
strated  their  maneuvering  performance  characteristics;  the  YF-22 
achieved  controlled  flight  at  a  60-degree  high  angle  of  attack. 

Some  members  of  the  Government  were  concerned  about  the  loca¬ 
tion  of  the  missile  bays  and  the  engine  inlets  of  the  YF-22.  They  feared 
that  when  a  missile  was  shot  from  the  aircraft,  the  aircraft  engines 
would  ingest  smoke  from  the  missile  and  flame  out.  To  demonstrate 
that  this  would  not  happen,  the  Lockheed  team  launched  an  AIM-9 
Sidewinder  missile  auid  an  AIM- 120  Advanced  Medium  Range  Air-to- 
Air  Missile  (AMRAAM)  from  the  missile  bays  of  the  the  YF-22. 

The  competing  contractor  teams  submitted  data  from  flight  test¬ 
ing  to  the  Government  as  part  of  their  proposal  for  the  EMD  phase 
on  January  2,  1991.  The  flight  testing  of  both  the  YF-22  and  the 
YF-23  convincingly  demonstrated  high  maneuverability  in  a  low- 
observable  fighter  and  the  ability  to  achieve  supersonic,  level  flight 
without  the  use  of  afterburner  (supercruise). 

Full-Scale  Radar  Cross-Section  Pole  Models 

The  ability  to  accurately  model  radar  cross-sections  (which  pro¬ 
duces  an  aircraft  radar  “echo”)  on  a  computer  had  not  been  satisfacto¬ 
rily  verified.  To  prove  the  stealth  features  of  their  designs,  both  con¬ 
tractors  developed  a  full-scale  model  aircraft  based  on  their  preferred 
system  concept  (the  design  the  team  would  propose  as  the  ATF  opera¬ 
tional  configuration).  The  models  (termed  pole  models  because  they 


Table  6-1 .  ATF  Prototype  Air  Vehicle  Test  Summary 

Prototype 

Sorties 

Flight  hours 

YF-22  (Lockheed/Boelng/General 

Dynamics) 

1  (General  Electric  engine) 

43 

52.8 

2  (Pratt  &  Whitney  engine) 

31 

38.8 

YF-23  (Northrop/McDonnell  Douglas 

1  (General  Electric  engine) 

34 

44.4 

2  (Pratt  &  Whitney  engine) 

m 

21.4 

TOTAL 124 

157.4 

Pfioto  courtesy  of  Lockheed-Marttn. 


A  YF-22  launches  an  unarmed  AIM-120  AMRAAM  missile. The  launch  of  the  AIM- 
120,  as  well  as  an  AIM-9  Sidewinder  missile,  proved  that  ingestion  of  the  smoke 
from  the  missiles  was  not  a  problem  for  the  YF-22  engines. 
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would  be  mounted  on  poles)  included  radar-absorbing  structures  and 
materials,  antennas,  simulated  engines,  simulated  canopy,  and  the 
cracks  and  gaps  caused  by  the  aircraft  flaps  and  other  aerodynamic 
control  surfaces.  The  contractors  conducted  more  than  800  hours  of 
tests  on  these  models  against  various  radar  emitters,  and  the 
Government  conducted  an  additional  400  hours  of  radar  cross-section 
tests.  The  contractors  also  carried  out  12,000  hours  of  radar  range  test¬ 
ing  on  model  components.  The  tests  verified  that  radar  cross-section 
test  results  could  be  predicted — pretest  predictions  were  usually  very 
close  to  the  actual  results.  The  data  from  these  ATF  pole  models  gave 
the  contractors  and  the  Government  high  confidence  that  they  could 
achieve  the  radar  cross-section  values  for  the  operational  aircraft. 

Test  Engines 

Capitalizing  on  lessons  learned  from  the  F-15,  F-16,  and  other 
jet  aircraft  programs,  the  Air  Force  knew  that  engine  development 
had  to  precede  airframe  design  and  that  the  aircraft-engine  inlet 
interface  had  to  be  designed  early  in  the  program.  Work  on  what  was 
to  become  the  Pratt  &  Whitney  F119  and  the  General  Electric  F120 
officially  got  underway  in  1983  when  the  Air  Force  propulsion  pro¬ 
gram  office  awarded  a  contract  to  each  engine  company  for  what 
was  initially  called  the  JAFE  program.  The  objectives  of  this  program 
were  increased  thrust  (with  a  goal  of  providing  supersonic  cruise 
without  the  use  of  afterburner),  fuel  efficiency,  improved  reliability, 
and  ease  of  maintenance. 

In  the  initial  phase,  both  engine  contractors  conducted  numerous 
component  rig  tests  on  key  sections  of  the  engine  and  tested  two 
ground  demonstrator  engines.  Under  the  renamed  ATF  Engine 
Program,  both  contractors  produced  six  prototype  engines  to  support 
the  YF-22  and  YF-23  flight  tests,  which  completed  over  4,000  hours  of 
ground  testing  and  accumulated  over  300  hours  of  flight  test.  For  the 
YF-22,  which  Lockheed  had  designed  to  use  two-dimensional  thrust 
vectoring  (the  direction  of  thrust  can  angle  up  or  down  20  degrees, 
increasing  the  agility  of  the  aircraft),  the  engines  completed  over 
1,500  hours  of  ground  testing  with  the  thrust- vectoring  nozzles. 

Flight  testing  ultimately  proved  that  the  F119  and  F120  engines 
could  generate  sufficient  thrust  to  maintain  supersonic  flight  with¬ 
out  afterburner  and  verified  the  engines’  improved  fuel  efficiency 
and  greatly  improved  reliability. 

Avionics  Prototypes 

In  the  mid-1980s  the  Air  Force  established  several  programs  in  its 
Avionics  Laboratory  to  exploit  the  new  technological  capabilities  of 


PiKJto  courtesy  of  United  Technalogoie^Pmtl  &  Whttney. 


A  feature  of  the  Ft  19  engine  is  its  ^ility  to  angle  the  thrust  of  the  engine  20  degrees  up  or  down.  The  high  thrust,  fuel  efficient 
engine  uses  fewer  and  less  complex  parts  than  previous  fighter  aircraft  engines. 
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aircraft  electronics.  The  programs  examined  improved  methods  of 
designing  radars,  electronic  warfare  systems,  communications 
systems,  and  overall  avionics  architecture  for  an  aircraft.  All  of  these 
greatly  improved  both  performance  and  the  fighter  pilot’s  situational 
awareness.  The  Avionics  Laboratory  programs  produced  six  compo¬ 
nents  that  proved  critical  in  the  development  of  the  ATF.  These  com¬ 
ponents  are  described  below: 

Pave  Pillar:  An  integrated  system  architecture  that  provides  a 
common  signal  processor,  data  bus,  and  other  avionics  elements 
for  functions  such  as  communications  and  radar 

Common  Signal  Processor:  A  computer  module  designed  for  the 
Pave  Pillar  architecture  to  provide  computational  power  for  mul¬ 
tiple  avionics  functions  including  communications,  electronic 
warfare,  and  radar  processing 

Ultra  Reliable  Radar:  A  phased-array  radar  with  numerous, 
small  transmit-and-receive  modules  coupled  to  improve  radeir 
performance  and  reliability 

Very  High  Speed  Integrated  Circuits  (VHSIC):  Electronic  com¬ 
ponents  that  support  higher  speed  and  more  powerful  micro¬ 
processors 

Integrated  Communication,  Navigation,  Identification  Avionics 
(ICNIA):  A  highly  integrated  suite  of  avionics  that  provides  a 
wide  array  of  systems  and  functions,  including  radios 
(VHFAJHF),  data  links  (such  as  the  Joint  Tactical  Information 
Distribution  System),  identification  friend  or  foe  (Mk  XII),  and 
location  (Global  Positioning  System) 

Integrated  Electronic  Warfare  System  (INEWS):  Similar  in  con¬ 
cept  to  the  ICNIA  and  designed  to  operate  within  the  Pave  Pillar 
architecture;  a  laboratory  system  that  analyzes  enemy  radio  and 
radar  signals  to  identify  the  t3T)e  of  threat  and  provide  appro¬ 
priate  countermeasures. 

I  can’t  think  of  a  set  of  technology  programs  that  has  ever 
provided  more  bang  for  the  buck  than  the  Air  Force  is  going 
to  get  from  Pave  Pillar,  VHISC,  ICNIA,  and  INEWS. 

Sherman  Mullins,  Lockheed  ATF  Program  Manager 
in  "The  ATF:  Hot  and  Stealthy,"  Air  Force  Magazine 

The  common  thread  of  these  technologies  is  the  integrated  manner  in 
which  they  were  to  be  designed  and  implemented.  However,  individual 


Rwto  awrtey  ol  Aiwaft 

Tliis  model  of  a  Boeing  757  shows  some  of  the  modifications  that  converted  the  aircraft  Into  an  avionics  ft^ng  laboratory  used  to 
develop  the  F-22’s  sensors/radms. 
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avionics  developments  were  hard  enough,  and  many  believed  integrated 
avionics  would  be  extremely  risky  Tb  quantify  and  reduce  the  risk,  the 
contractors  developed  an  Avionics  Ground  Prototype  and  an  Avionics 
Flying  Laboratory,  also  known  as  the  Flying  Ifest  Bed.  The  Avionics 
Ground  Prototype  incorporated  brassboard  hardware  and  commercial 
processors  and  test  eqmpment  to  demonstrate  key  technologies.  The 
primary  objective  of  the  Avionics  Flying  Laboratory  was  to  test  brass- 
board  sensors  and  processors  while  airborne. 

As  part  of  the  avionics  testing,  the  contractors  built  and  tested  30 
major  avionics  module  types  and  over  650  brassboard  modules.  As 
part  of  the  effort,  the  contractors  wrote  over  1.4  million  lines  of  Ada 
software  code.  To  support  software  development,  the  contractors 
developed  a  prototype  software  development  "environment"  that 
provided  standard  equipment  and  tools  and  interlinked  over  600  soft¬ 
ware  programmers  in  six  different  states. 

The  contractor  teams  also  tested  additional  types  of  hardware. 
Each  team  developed  a  prototype  electrically  scanned  antenna  array 
to  prove  new  radar  technology.  As  part  of  this  effort,  the  radar  sub¬ 
contractors  produced  more  than  2,000  transmit/receive  models  for 
component  radar  system  testing.  The  communications  subcontractor 
for  each  team  developed  a  communications,  navigation,  and  identifi¬ 
cation  suite  that  simultaneously  processed  signals  from  the  commu¬ 
nications  radios,  radio  navigation  aides,  Global  Positioning  System, 
Joint  Tactical  Information  Distribution  System,  and  Mark  XII  iden¬ 
tification  friend-or-foe  system. 

The  competing  teams  integrated  and  demonstrated  elements  of 
the  avionics  test  hardware  in  the  Avionics  Ground  Prototype  labora¬ 
tory  and  then  flew  some  components  in  the  Fl5dng  Test  Bed  (a  Boeing 
757  for  the  YF-22  team,  a  BAC-111  for  the  YF-23  team).  As  part  of  the 
final  ground  protot3pe  testing,  both  contractors  used  breadboard 
processors  programmed  with  Ada  software  code  to  process  information 
from  signals  captured  by  developmental  antennas  and  processors  car¬ 
ried  by  the  fl3dng  test  beds,  resulting  in  an  integrated  presentation  of 
data  for  the  pilot.  The  YF-22  team  also  flew  the  breadboard  processors 
and  demonstrated  in-flight  operation  of  their  integrated  suite. 

Reliability,  Maintainability,  and  Support 

From  the  beginning,  Tactical  Air  Command  emphasized  that  the 
ATF  was  to  have  significantly  increased  reliability  and  require  less 
support  than  the  F-15.  Design  for  increased  reliability  and  ease  of 
maintenance  started  during  the  concept  development  investigation 
phase.  The  contractors  verified  the  new  design  features  and  technolo¬ 
gy  diiring  the  demonstration/validation  phase. 
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Both  contractor  teams  checked  the  installation  and  accessibility 
of  both  the  F119  and  F120  engines  within  a  mockup  of  the  aircraft 
engine  bay,  tested  the  ease  of  gaining  access  to  major  components 
installation  and  repair,  and  tested  whether  the  air-to-air  missiles 
(AIM-9  and  AIM-120)  would  fit  within  the  weapon  bays.  Both  teams 
demonstrated  the  basic  concepts  of  their  integrated  diagnostic 
systems  and  generated  initial  paperless  (digital)  technical  orders  for 
repair  and  maintenance  of  the  aircraft. 

The  reliability  engineers  also  conducted  a  continuing  logistical 
support  analysis  that  identified  critical  failure  mechanisms.  After 
discussion  with  the  program  office  and  the  users,  the  contractor 
design  engineers  revised  the  aircraft  and  their  component  designs 
based  on  the  analysis.  As  a  result  of  reliability  demonstration  and 
analysis,  the  contractors  incorporated  over  1,000  design  changes  to 
improve  the  reliability  and  ease  of  maintenance  of  the  weapon  system. 

Materials 

In  the  early  1980s  the  aviation  community  held  high  hopes  that 
advanced  composite  materials  could  replace  much  of  the  metal, 
primarily  aluminum,  in  the  aircraft.  Tb  measure  the  physical  proper¬ 
ties  of  advanced  materials,  including  composites,  and  see  how  they 
would  perform  in  different  environments  and  configurations,  the  com¬ 
peting  contractors  conducted  over  9,000  tests  on  various  materials. 

Materials  testing  had  a  high  payoff.  For  example,  one  t3^e  of 
composite  material,  called  thermoplastics,  looked  very  promising  at 
the  start  of  demonstration/validation.  Thermoplastics,  formed  under 
high-pressure  heating,  can  be  reprocessed  again  and  again,  making 
them  a  good  choice  for  items  requiring  repair  in  the  field.  Another 
type  of  composite  material,  called  thermosets,  undergoes  a  one-time 
chemical  change  when  it  is  heated  and  cured.  This  property  makes 
manufacturing  errors  more  significant  (because  the  piece  can’t  be 
reworked)  and  requires  different  repair  procedures  in  the  field, 
Lockheed  predicted  that  thermoplastics  would  be  the  major  type  of 
composite  for  the  production  aircraft. 

After  4  years  of  development  and  testing,  both  contractors  deter¬ 
mined  that  thermoplastics  did  not  measxire  up  to  predicted  perfor¬ 
mance  and  cost.  However,  advanced  thermoset  materials  performed 
far  better  than  expected.  As  a  result,  Lockheed  will  use  thermoplas¬ 
tics  only  for  a  few  specialty  parts,  such  as  those  subject  to  impact  with 
foreign  objects  (belly  skins  and  removable  panels)  and  will  use 
advanced  thermosets  as  the  primary  composite  material.  As  a  direct 
result,  the  Lockheed  team  entered  the  EMD  phase  with  a  known 
family  of  composites  to  use  in  the  production  aircraft. 
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Modeling  and  Simulation 

Both  contractor  teams  used  computer  modeling  in  every  aspect 
of  maturing  the  ATF  technology.  The  models  fall  into  two  broad 
categories:  design  models  and  effectiveness  models.  The  ATF  teams 
used  them  to  develop  and  refine  requirements,  assess  the  applicabili¬ 
ty  of  technology,  and  establish  the  weapon  system  design.  The  ATF 
program  also  used  man-in-the-loop  simulations  to  verify  require¬ 
ments  and  cockpit  layout  and  to  evaluate  proposed  capabilities  and 
information  presentation. 

Design  models.  These  models,  which  include  aerodynamic, 
structural,  and  avionics  models,  predict  performance  and  can  extrap¬ 
olate  test  results.  Their  output  serves  as  input  for  the  effectiveness 
models.  The  development  teams  used  design  models  iteratively. 
Their  initial  use  predicted  performance,  and,  as  testing  )delded  data, 
the  teams  updated,  refined,  and  validated  the  design  models.  Once 
the  models  were  validated,  the  design  engineers  could  use  them  to 
extrapolate  test  results  and  predict  a  fuller  range  of  performance 
for  the  weapon  system. 

Aerodynamic  models:  The  design  teams  used  aerodynamic  models  to 
predict  such  information  as  aircraft  range,  maneuverability,  rate 
of  turn,  and  external  loads.  The  contractors  used  data  from  more 
than  30,000  hours  of  wind  tunnel  testing  and  157  hours  of  flight 
testing  to  refine  and  validate  the  models. 

Structural  models:  Component  design  engineers  use  structural 
models  to  predict  the  performance  of  a  specific  item,  such  as  a 
spar.  For  example,  these  models  can  predict  internal  loads  within 
the  aircraft,  which  the  engineering  team  can  then  use  to  size  the 
spar.  The  models  can  predict  the  performance  of  the  spar  for  var¬ 
ious  configurations  and  determine  its  ultimate  strength  and 
fatigue  life.  Important  input  variables  for  the  structural  models 
are  the  material’s  physical  properties  (such  as  shear  strength, 
directional  strength,  and  stiffness  factor).  The  design  teams  used 
results  from  material  coupon  testing  in  the  structural  models  to 
determine  the  type  of  material  to  use  for  a  specific  component. 

Avionics  model:  Avionics  engineers  used  models  to  predict  the  perfor¬ 
mance  of  such  items  as  radars,  radios,  and  antennas  and  the  most 
effective  location  for  antennas  and  other  sensors. 

The  use  of  the  avionics  model  in  correctly  sizing  the  radar  anten¬ 
na  shows  the  complexity  of  the  design  process  for  an  aircraft  and  the 
importance  of  models.  Radar  performance  depends  on  many  things, 


Photo  courtesy  of  LadtiwcHlerin. 

Models  of  the  YF-22  prototype  were  used  in  wind  tunnel  tests  to  predict  wind  resistance  to  the  requirerrrent  that  the  aircraft  be  able 
to  sustain  supersonic  cruising  (supercruise)  speeds. 
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including  the  ability  of  its  transmitter  and  receiver  as  well  as  the 
shape  and  size  of  the  radar  antenna  (in  general,  the  larger  the  size, 
the  farther  away  a  radar  can  detect  targets).  Therefore,  the  radar 
engineer  wants  the  antenna  aperture  to  be  as  large  as  possible. 
However,  the  radar  must  be  mounted  on  the  front  of  the  fighter 
(just  behind  the  pointed  radome).  A  large  radar  drives  the  diameter  of 
the  fuselage,  and  a  wide  fuselage  increases  the  size,  drag,  and  radar 
signature  of  the  aircraft.  Thus  the  airframe  engineers  and  engineers 
concerned  with  the  radar  signature  of  the  aircraft  want  to  minimize 
the  size  of  the  fuselage. 

To  balance  radar  performance,  aerodynamic  performance,  and 
low  radar  return,  the  design  team  used  the  respective  models.  The 
radar  engineer  computed  the  smallest  shape  of  radar  that  could 
meet  the  requirements  for  identif5ring  targets  (like  enemy  aircraft) 
at  a  certain  distance.  The  airframe  designers  and  low-observability 
engineers  computed  the  largest  fuselage  diameter  that  would  still 
meet  their  requirements.  This  iterative  process  continued  until  the 
design  team  determined  the  optimum  size  for  the  fuselage.  As  test 
results  came  in  from  the  radar  laboratories  and  wind  tunnels,  the 
design  engineers  updated  their  models,  refined  their  calculations, 
and,  if  needed,  updated  the  preferred  system  concept  for  the  radar 
and  aircraft  fuselage. 

Effectiveness  models.  The  program  office,  the  contractors, 
and,  in  some  cases,  TAG  used  effectiveness  models  to  set  perform¬ 
ance  requirements  and  conduct  trade-offs.  Examples  include:  TAG 
BRAWLER,  Enhanced  Surface-to-Air  Missile  Simulation  (ESAMS), 
and  the  Logistics  Gomposite  Model  (LGOM). 

TAG  BRAWLER:  This  complex  computer  model  predicts  the  out¬ 
comes  of  friendly  versus  enemy  air  engagement  by  modeling 
friendly  fighters,  the  ATF,  and  enemy  fighters.  BRAWLER  can 
model  a  single  fighter  against  a  single  fighter,  many  fighters 
against  many  fighters,  or  a  combination.  During  demonstra¬ 
tion/validation  the  ATF  program  completed  over  22,000  TAG 
BRAWLER  runs  to  size  the  aircraft,  establish  its  maximum 
radar  cross-section,  and  ultimately  evaluate  the  military  effec¬ 
tiveness  of  the  design.  The  team  also  used  the  model  to  establish 
such  features  as  the  number  and  type  of  air-to-air  missiles  the 
aircraft  needed  to  carry. 

Both  contractors  and  the  Grovemment  conducted  evaluations 
and  trade-offs  using  TAG  BRAWLER.  In  fact,  it  is  an  understate¬ 
ment  to  say  that  the  contractors  defined  the  ATF  design  in  large 
part  based  on  the  results  of  TAG  BRAWLER  runs.  During  the 
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design  process  the  contractor  teams  would  change  some  aspect  of 
their  preferred  system  concept  and  submit  the  results  of  their 
TAG  BRAWLER  runs  to  the  program  office  for  evaluation.  The 
program  office  would  then  run  the  changed  design  on  their  own 
version  of  BRAWLER  to  validate  the  contractor  performance  pre¬ 
dictions.  This  procedure  verified  the  accuracy  of  the  program 
office  and  the  contractor  calculations,  serving  as  a  system  of 
checks  and  balances  for  the  modeling. 

Because  the  contractors  and  the  Government  had  to  compare 
data  from  their  separate  runs,  they  had  to  use  TAG  BRAWLER 
models  that  were  of  the  same  configuration  and  that  were  vali¬ 
dated.  After  the  Air  Force  Office  of  Studies  and  Analysis  had 
verified  the  TAG  BRAWLER  model  and  validated  its  accuracy,  it 
provided  a  level  playing  field  on  which  the  Government  could 
compare  the  predicted  performance  of  one  contractor  design  with 
the  other.  Both  contractors  submitted  TAG  BRAWLER  data  as 
part  of  their  proposal  for  the  EMD  phase. 

Enhanced  Surface-to-Air  Missile  Simulation:  The  ESAMS  model 
compared  the  ATF  radar  signature  with  enemy  missile  systems  to 
determine  the  ability  of  a  missile  to  acquire  an  ATF  and  shoot  it 
down.  The  teams  used  ESAMS  data  to  see  how  well  the  ATF  could 
penetrate  an  integrated  air  defense  system  with  current  and 
projected  surface-to-air  missiles.  This  data  helped  finalize  the 
ATF  radar  cross-section  requirements.  As  it  did  with  TAG 
BRAWLER,  the  Air  Force  validated  and  verified  the  ESAMS  and 
used  it  to  compare  the  contractor  designs. 

Logistics  Gomposite  Model  (LGOM):  The  Air  Force  has  traditionally 
used  the  LGOM  to  predict  the  manpower  required  to  support  a 
new  weapon  system.  The  Air  Force  modified  the  LGOM  so  that  it 
could  not  only  forecast  manpower  but  also  determine  the  likeli¬ 
hood  of  the  aircraft  being  available  to  fly  a  mission.  The  LGOM 
also  predicted  failure  rates  for  most  aircraft  subsystems  and 
components.  With  these  rates  and  known  maintenance  require¬ 
ments,  the  development  teams  computed  what  supplies  and 
equipment  an  operational  fighter  squadron  would  need  for  a  peri¬ 
od  of  time.  That  information  and  another  model,  the  Gomputer- 
Aided  Load  Manifest  (GALM)  model  helped  the  teams  determine 
how  many  G-141  cargo  aircraft  it  would  take  to  airlift  the  needed 
supplies  and  equipment.  The  LGOM  also  predicted  the  sortie 
generation  rate  (the  ability  of  the  aircraft  to  fly  missions). 


Photo  Eric  Schuizirtpr  courts  of  Lo«»©d-Mariii. 


Man-ln-the-loop  testing  was  essential  in  the  F-22  demonstiatior^iidation  phase.  Simulators,  such  as  this  domed  system, 
allowed  pilots  to  “fl/the  F-22  before  the  YF-22  proto^pe  was  built.  The  simulations  used  the  cxsntractors  system  concept  to 
determine  what  pilots  really  needed  in  the  n^  weapon  system,  a  step  which  was  invaluaWe  In  dweloping  the  prototype. 
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Man-in-the-Loop  Simulations.  The  ATF  demonstration/valida¬ 
tion  phase  was  unusual  among  programs  in  the  degree  of  user 
involvement.  As  part  of  the  requirements  iteration  process,  the  users 
needed  a  tool  to  help  them  fully  understand  possible  features  and 
capabilities  and  determine  their  utility  to  the  combat  pilot.  Both 
contractor  teams  established  manned-flight  simulators  to  allow  TAG 
pilots  to  fly  the  aircraft  that  incorporated  the  contractor  preferred 
system  concept. 

The  simiilators  had  the  ability  to  model  different  types  of  avionics 
equipment  (how  useful  is  an  infrared  search-and-track  system),  types 
of  aircraft  performance  features  (how  often  does  a  pilot  need  speed 
brakes),  and  types  and  quantities  of  weapons  (how  often  does  a  pilot 
of  a  stealth  fighter  need  to  fire  a  short-range  infrared  missile).  During 
man-in-the-loop  simulations  the  pilots  began  to  develop  new  tactics 
that  took  advantage  of  the  various  features  of  the  new  fighter  (includ¬ 
ing  stealth,  advanced  avionics,  and  thrust-vectored  and  supercruise 
flight),  which  helped  refine  aircraft  requirements  and  uncovered  new 
requirements  that  maximized  the  fighter’s  capability  ageiinst  enemy 
aircraft  and  surface-to-air  missiles. 

As  a  result  of  more  than  800  man-in-the-loop  simulations,  the 
users,  the  program  office,  and  the  contractors  had  a  much  better 
understanding  of  what  the  fighter  needed  to  do  and  what  it  could  do. 

Results 

The  ATF  program  objective  in  the  demonstration/validation  phase 
was  to  mature  and  prove  the  technology  needed  to  design  the  weapon 
system.  Did  it  meet  its  objective? 

The  ATF  program  invested  more  than  4  years  in  demonstra¬ 
tion/validation  with  two  highly  talented  aerospace  teams  working 
out  all  aspects  of  technical  risk.  As  a  result  of  the  rigorousness  of 
the  demonstration/validation  phase,  in  more  than  4  years  of  EMD,  the 
F-22  program  has  not  run  into  any  significant  technical  problems — 
a  remarkable  accomplishment  for  a  complex  weapon  system  that 
incorporates  a  great  deal  of  new  technology. 

Was  the  addition  of  the  flying  prototypes  helpful  to  the  program? 
The  ATF  foimders  say  yes — ^but  for  different  reasons.  Fain  believes 
that  the  benefit  of  having  the  contractors  design  gtnd  construct  the 
prototype  was  that  it  forced  them  to  predict  the  prototype  perfor¬ 
mances  and  thus  to  validate  their  models,  which  improved  their  abil¬ 
ity  to  properly  design  the  production  aircraft. 

The  other  founders,  Bucher,  Abell,  and  Graves,  see  the  benefit  of 
the  prototypes  as  being  more  procedural  than  technical.  Making  the 


Managing  the  Technology  75 


multi-company  teams  design  and  build  a  test  aircraft  forced  them  to 
create  a  business  and  working  arrangement  that  allowed  multicom¬ 
pany  design  and  construction  to  take  place.  This  prototype  process 
saved  significant  time  in  the  EMD  phase  because  the  hard  business 
decisions  had  already  been  made  and  the  procedures  had  been  devel¬ 
oped  and  refined. 

In  Fain's  view,  thanks  to  the  demonstration/validation  effort,  the 
F-22  program  advanced  technically  in  the  areas  of  low  observability, 
engine  performance,  and  avionics.  However,  the  program  also  learned 
that  it  could  not  go  as  far  as  originally  thought  in  the  area  of  aerody¬ 
namic  performance.  This  outcome  was  acceptable,  as  the  TAG 
BRAWLER  model  verified  that  the  keys  to  mission  effectiveness  are  low 
observability,  supercruise,  and  advanced  avionics. 
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I  think  the  concept  of  integrated  product  teams  is  a 
breakthrough,  A  breakthrough  in  breaking  down  the 
barriers  between  functional  components  to  produce 
teamwork.  It  has  prevented  innumerable  problems 
from  creeping  into  the  [F-22J  program  that  would  have 
taken  months,  if  not  years,  to  uncover  had  we  not  had 
the  IPT  process. 


General  John  Michael  Loh, 
Commander,  Air  Combat  Command 


After  defining  the  requirements  and  managing  the  technologies, 
the  third  element  in  managing  risk  is  to  set  up  a  management 
system  focused  on  the  product.  This  chapter  reviews  the  man¬ 
agement  approach,  integrated  product  development  (IPD),  and  the 
management  structure,  integrated  product  teams  (IPTs)  (see  Figure 
7-1).  The  next  chapter  reviews  the  F-22  management  tool  set. 

Integrated  Product  Development:  A  Definition 

Within  much  of  the  DOD  and  defense  industry  the  phrase  “inte¬ 
grated  product  team”  (IPT)  is  used  for  cross-functional,  empowered 
work  groups  that  support  integrated  product  development  (IPD). 

The  Air  Force  defines  IPD  as  “a  team  approach  to  systematical¬ 
ly  integrate  and  concurrently  apply  all  necessary  disciplines 
throughout  the  system  life  cycle  to  produce  an  effective  and  efficient 
product  or  process  that  satisfies  customer  needs.”  In  1990,  the  ATF 
program  office  reviewed  a  study  conducted  at  Wright-Patterson  Air 
Force  Base,  Ohio,  entitled  The  Results  of  the  Aeronautical  Systems 
Division  Critical  Process  Team  on  Integrated  Product  Development, 
This  report  defined  IPD  as  follows: 

IPD  is  an  efficient  process  of  bringing  a  product  from 
user’s  needs  to  field  operation.  The  basic  principle  is  to 
iterate  and  integrate  the  design  of  a  product  and  design 
of  its  manufacturing,  operation,  support  and  training 
processes  with  specific  focus  on  achieving  low-cost 
development,  production,  operations  and  support  within 
the  shortest  schedule  while  achieving  robust  quality  of 
products  and  services. 

Fain’s  definition  of  IPD  was  more  succinct: 
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IPD  is  nothing  more  than  the  right  people,  at  the  right 
time,  focused  on  the  right  problems,  to  make  the  right 
decisions! 

The  Characteristics  of  IPD 

IPD  has  four  major  characteristics  (Figure  7-2). 

IPD  is  customer-focused.  The  reason  any  activity  exists  is  to 
provide  a  product  (or  a  service)  to  meet  a  customer's  needs.  IPD  starts 
with  understanding  the  customer’s  needs  and  working  to  meet  them. 

EPD  is  product-oriented.  IPD  redirects  the  old  focus  on  func¬ 
tional  expertise  to  the  item  that  will  meet  the  customer’s  needs — the 
product.  The  classic,  pre-IPD  approach  to  development  also  designed 
and  produced  products  but  emphasized  an  individual’s  functional 
expertise  (engineering,  manufacturing,  logistics)  rather  than  the 
product.  The  intent  behind  IPD  is  to  have  workers  emphasize  the 
product  first  and  their  technical  specialty  second. 

IPD  uses  empowered,  cross-functional  teams.  To  develop  a 
product  that  meets  customer  needs,  IPD  relies  on  a  group  of  individu¬ 
als  who  have  the  needed  skills  to  ensure  the  product  is  “doable”:  one 
that  manufacturing  can  affordably  build  and  that  meets  the  near-  and 
long-term  needs  of  the  customer.  IPD  relies  on  all  needed  specialties 
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making  design  decisions  together  to  yield  a  high-quality  product* 
Design  engineers  (e.g.,  electrical,  mechanical,  and  aeronautical)  work 
with  financial  and  manufacturing  personnel  (to  make  sure  the  design 
can  be  built  affordably)  and  logistics  experts  (to  make  sure  the  prod¬ 
uct  will  last  and  can  be  easily  fixed)  as  well  as  test  engineers  to  create 
a  product  that  will  do  what  the  customer  wants.  To  successfully  build 
their  product,  these  multidisciplinary  teams  must  make  decisions 
about  the  design  and  development  of  their  product.  Therefore,  the 
organization’s  leadership  must  give  the  teams  the  responsibility  and 
authority  to  make  those  decisions. 

IPD  takes  a  long-term  view.  The  IPD  teams  must  focus  on  the 
full  needs  of  the  customer.  They  must  meet  the  immediate  need  for  a 
product  that  does  what  the  customer  wants,  but  as  mentioned  above, 
it  must  be  easy  to  fix  when  it  breaks.  The  product  should  ideally  be 
easy  to  modify  and  improve  in  the  future  to  meet  the  evolving  needs 
of  the  customer.  This  requires  the  design  team  to  think  about  the 
product  from  delivery  through  modification  to  disposal. 


Rgure  7-2.  Key  Elements  of  Integrated  Product  Development 
Integrated  Product  Development 

•  Is  customer-focused 

•  Is  product-oriented 

•  Uses  empowered,  cross-functional  teams 

•  Takes  a  long-term  view 


The  History  of  IPD 

IPD  traces  its  origins  back  to  the  engineering  processes  first  used 
in  the  1950s  by  the  Japanese  in  their  “lean  production”  method  of 
automobile  manufacturing  (see  Figure  7-3).  The  philosophy  behind 
lean  production  was  to  deliver  high-quality  products  and  to  continu¬ 
ously  improve  the  processes  used  to  develop  those  products.  Japanese 
automakers  achieved  continuous  improvement  by  using  collocated, 
multifunctional  teams  that  concentrated  on  designing  for  cost-effec¬ 
tive  production  of  products  that  met  customer  requirements. 

In  the  1970s,  the  term  concurrent  engineering  emerged  to 
describe  a  process  similar  to  (and  based  on)  Japanese  lean  produc¬ 
tion.  Concurrent  engineering  called  for  a  multifunctional  team  that 
considered  design  as  well  as  manufacturing  and  affordability  in 
the  product  initial  design  phase.  In  the  1980s,  thanks  in  part  to 
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competitive  pressure  from  Japan,  several  sectors  of  U.S.  industry 
(most  notably  car  manufacturers)  adopted  concurrent  engineering 
design  practices. 

In  1988,  the  Institute  for  Defense  Analyses  studied  the  applica¬ 
bility  of  concurrent  engineering  to  the  weapon  system  acquisition 
process.  Shortly  after,  the  ATF  program  office  decided  to  imple¬ 
ment  concurrent  engineering  in  the  engineering  and  manufacturing 
development  (EMD)  phase.  In  1990,  the  Air  Force  Systems 
Command  leadership,  as  part  of  its  quarterly  leadership  Horizon 
conference,  set  the  goal  of  implementing  IPD.  The  Commander  of 
Air  Force  Systems  Command,  General  Ronald  W.  Yates,  believed  in 
the  benefits  of  IPD  but  needed  to  figure  out  how  to  implement  it 
command-wide. 

Integrated  Product  Development  in  the  F-22  Program 

Prior  to  the  beginning  of  F-22  engineering  and  manu¬ 
facturing  development,  the  Air  Force  acquisition  communi¬ 
ty  had  talked  about  integrated  product  development,  that 
was  a  theme  we  had  in  the  late  1980s.  But  we  had  not 
figured  out  how  to  implement  it.  We  had  not  gotten  down 
to  the  level  of  integrated  product  teams.  That  is  something 
clearly  pioneered  by  the  F-22  and  proven  to  be  very  valu¬ 
able.  As  I  sit  here  in  1996,  just  5  years  after  the  start  of 
F-22  EMD,  I  am  amazed  how  the  IPT  concept  has  spread. 

I  met  with  a  contractor  this  morning  who  explained  his 
teaming  arrangements  for  a  new  program.  His  team  has 
all  of  the  functionals,  all  from  various  companies,  collocat¬ 
ed  in  the  same  building.  That  was  unheard  of  in  the  1980s. 

That  was  unheard  of  in  1990.  The  success  of  the  IPTs  has 
spread  throughout  the  industry  and  to  all  of  the  other 
major  programs  throughout  the  Air  Force.  That  is  all  due 
to  the  F-22. 


Darleen  A.  Druyim,  Principal  Deputy 
Assistant  Secretary  for  Acquisition  Management 

The  implementation  of  IPD  for  the  F-22  program  evolved  natural¬ 
ly  from  the  founders'  principles  (such  as  teamwork,  integration,  own¬ 
ership,  and  long-term  view)  and  several  experiences  in  the  ATF 
demonstration/validation  phase. 

During  demonstration/validation,  the  program  office  established 
working  groups  made  up  of  personnel  from  the  Air  Force,  the  Navy, 
and  the  rest  of  the  DOD  to  address  a  full  spectrum  of  issues,  includ¬ 
ing  armament  integration,  security  management,  common  avionics, 
and  cost  estimation.  Following  the  fourth  principle,  involving  every- 
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Figure  7-3.  History  of  Integrated  Product  Development 
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one  who  has  a  stake  in  the  outcome,  these  working  groups  included 
representatives  from  not  only  many  organizations  but  also  many 
disciplines.  Through  participation  in  these  groups,  the  various  organ¬ 
izations  could  raise  their  issues  and  debate  the  various  suggestions 
from  the  others.  The  members  knew  and  understood  the  reasons 
behind  the  decisions.  Once  given  a  chance  to  voice  their  opinions  and 
thus  be  part  of  the  decisionmaking  process,  they  could  effectively 
explain  back  at  their  home  organizations  why  the  ATF  program  (of 
which  they  were  a  part)  had  made  a  certain  decision. 

Another  example  of  effective  teaming  during  the  demonstration/ 
validation  phase  came  in  the  form  of  tiger  teams,  which  the  program 
office  set  up  when  some  part  of  the  program  ran  into  difficulties. 
These  tiger  teams  consisted  of  the  people  with  the  right  expertise  to 
solve  the  problem.  For  example,  the  tiger  team  for  a  business  problem 
might  include  individuals  from  contracts,  finance,  and  program 
management.  Often  these  teams  had  members  from  various  Govern¬ 
ment  groups  as  well  as  from  the  appropriate  contractors.  Because  the 
tiger  teams  included  people  with  the  necessary  expertise  and  a  stake 
in  the  outcome,  the  solution  usually  fell  within  everyone^s  constraints 
and  met  everyone's  expectations.  Fain  found  that  the  tiger  teams 
delivered  highly  successfiil  strategies  and  solutions. 
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The  example  that  convinced  Fain  that  the  concept  of  multifunc¬ 
tional  teams  had  a  high  payoff  came  from  the  Lockheed  YF-22 
design  effort.  In  1987  and  early  1988,  the  Lockheed-Boeing-Greneral 
Dynamics  team  could  not  agree  on  the  design  for  its  prototype 
version  of  the  ATR  To  meet  the  scheduled  flight  date  of  1990,  the 
YF-22  team  had  to  quickly  finalize  the  design.  After  repeated  failures 
to  reach  agreement,  the  Lockheed  program  manager,  Sherman 
Mullins,  brought  the  key  design  engineers  and  manufacturing  per¬ 
sonnel  into  one  location,  telling  them  that  no  one  could  go  home 
until  they  reached  an  agreement.  The  process  worked.  The  design 
team  emerged  several  weeks  later  with  a  solid  design  that  each  com¬ 
pany  felt  good  about. 

Late  in  demonstration/validation,  as  the  founding  fathers 
planned  the  EMD  phase.  Fain  asked  Abell,  Bucher,  and  Graves  a 
simple  question:  “Today  we  always  use  tiger  teams  when  we  get  in 
trouble.  For  EMD,  why  don’t  we  use  permanent  tiger  teams?”  As  the 
foimders  considered  the  concept  of  permanent  tiger  teams,  they 
discussed  the  evolution  of  the  front  office  group  (FOG)  decision 
process,  in  which  Fain  would  call  his  FOG  together  as  a  team  to  cre¬ 
ate  a  strategy  and  implement  a  solution  that  satisfied  each  member’s 
functional  constraints.  The  foimders  decided  to  push  this  concept  to 
the  lowest  levels.  The  result,  the  concurrent  leadership  described  in 
chapter  3,  was  the  establishment  of  multifunctional  teams  (just  like 
the  FOG)  as  permanent  tiger  teams. 

To  remind  the  teams  to  maintain  their  focus  on  the  product,  the 
program  office  originally  called  these  teams  IPD  teams.  During  one 
of  the  early  organizational  planning  meetings.  Colonel  Peter  Smith, 
Fain’s  program  manager  for  the  engine  program,  asked,  “If  we  are 
focusing  on  the  entire  life  cycle  [a  long-term  view]  and  not  just  the 
up-front  design,  why  do  we  put  the  emphasis  on  development? 
Shouldn’t  this  team  be  around  during  production  and  perhaps  even 
to  support  the  jet?”  Smith’s  logic  hit  home.  Fain  dropped  “develop¬ 
ment”  and  changed  the  name  to  integrated  product  team. 

Team  Organization 

Figure  7-4  shows  the  structure  of  the  F-22  program  office  and 
the  way  it  implemented  IPTs.  The  program  office  has  four  major 
integrated  product  teams:  Air  Vehicle,  Engine,  Support  System,  and 
Training  System,  each  with  a  number  of  subintegrated  product 
teams  that  focus  on  lower  level  products.  For  example,  the  Air 
Vehicle  team  includes,  among  others,  the  Armament,  Avionics, 
Cockpit,  and  Utilities  and  Subsystems  IPTs.  Because  of  the  com¬ 
plexity  of  some  of  the  products,  each  team  is  further  broken  down 


Figure  7-4.  The  F-22  System  Program  Office 
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into  smaller  subintegrated  product  teams  (for  example,  avionics  sub¬ 
teams  include  Radar,  Core  Processor,  and  Offensive  Avionics). 

The  program  office  also  includes  functional  offices  such  as  logis¬ 
tics,  systems  engineering,  and  test.  However,  the  composition  of 
these  offices  is  much  different  than  in  a  traditional  program  office. 
At  the  Air  Force  Aeronautical  Systems  Center,  the  Director  of 
Engineering  would  typically  direct  around  half  of  the  program  office, 
150  people.  In  the  F-22  program  office,  the  engineers  are  located  in 
the  product  teams  (e.g.,  Air  Vehicle,  Engine).  The  Directorate  of 
Engineering  now  has  fewer  than  20  individuals  assigned.  Clearly, 
the  focus  is  on  the  product. 

The  contractors  also  adopted  IPTs.  As  is  shown  in  Figure  7-5,  the 
Lockheed  F-22  program  and  the  F-22  program  office  have  very 
similar  organizational  structures — by  design.  The  program  office 
emulated  the  contractor  organization  to  support  teamwork  and  com¬ 
munications.  In  fact,  when  the  F-22  team  members  describe  their 
IPT  membership,  they  include  people  from  the  program  office,  the 
contractors,  and  Air  Combat  Command  (ACC).  Similar  organizations 
make  this  one-to-one  matching  much  easier. 

The  F-22  Integrated  Product  Teams 

Figure  7-6  shows  one  F-22  IPT,  Support  Data,  and  its  lower  tier 
IPTs.  This  team,  as  well  as  characteristics  of  IPD  as  they  relate  to 
such  F-22  IPTs,  are  described  below. 

The  Support  Data  IPT  is  one  of  four  teams  that  make  up  the 
F-22’s  third  major  team,  the  Support  Systems  IPT.  The  Chief  of  the 
Support  System  IPT  holds  the  sub-IPTs  responsible  for  cost,  sched¬ 
ule,  and  performance  in  the  development  of  their  products.  The 
Chief  of  the  Support  Data  IPT,  Major  Lowry,  serves  as  a  program 
director  for  all  elements  of  his  products.  For  example,  he  and  his 
contractor  counterpart  are  responsible  for  seeing  that  the  Integrated 
Management  Information  System  (a  portable  computer  that  flight- 
line  technicians  will  use  to  display  technical  orders  and  diagnose 
problems  with  the  aircraft)  meets  its  performance  requirements. 
They  also  see  that  the  design  team  meets  the  schedule  with  respect 
to  the  rest  of  the  F-22  program,  meets  its  producibility  cost  targets, 
and  does  not  exceed  its  annual  development  budget. 

Note  that  the  Support  Data  IPT  has  a  diverse  set  of  team 
members.  The  team  includes  managers,  engineers,  and  logistics 
experts  as  well  as  a  collocated  representative  from  ACC,  who  helps 
ensure  that  the  contractor  and  program  office  understand  the  needs  of 
the  command.  Although  located  on  the  Support  Data  team,  the  ACC 


Figure  7-5.  The  Lockheed  F-22  Program:  1991 
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representative  helps  in  all  areas  of  the  program  office  that  require 
operational  insight  and  skills  (e.g.,  flightline  maintenance). 

The  F-22  IPTs,  including  the  Support  Data  IPT,  incorporate  the 
characteristics  of  IPD  previously  discussed. 

They  are  customer-focused.  The  F-22  IPTs  are  focused  on 
their  customer,  Air  Combat  Command.  Members  of  ACC  are 
assigned  to  the  program  office  to  improve  the  communication  among 
the  program  office,  the  contractor,  and  ACC. 

They  are  product-oriented.  The  Support  Data  IPT  is  orga¬ 
nized  along  its  portfolio.  The  contractor  side  of  the  team  has  IPTs 
that  focus  on  lower  levels  of  indenture  as  well.  In  the  ever-increas¬ 
ing  tiers  of  teams,  the  F-22  Program  Director  has  an  IPT  responsi¬ 
ble  for  every  element  of  the  weapon  system.  Each  is  based  on  a 
product  or  group  of  products. 

They  are  empowered,  cross-functional  teams.  In  an  IPT, 
managers,  design  engineers,  manufacturing  engineers,  logisticians, 


Figure  7-6.  The  F-22  Program  Support  Data  IPT 
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and  in  some  cases  ACC  representatives  all  work  in  the  same  physical 
area  of  the  program  office.  A  dedicated  contracting  specialist  and  a 
financial  expert  are  also  assigned  to  the  team.  In  the  largest  teams 
(Air  Vehicle  and  Engines),  the  finance  specialists  sit  with  their 
teams.  The  contractor  representative  and  the  team  members  main¬ 
tain  contact  through  frequent  visits  to  each  other’s  facilities  and 
frequent  communication  by  telephone,  electronic  mail,  and  F-22 
dedicated  video  teleconferences. 

They  take  a  long-term  view.  Each  team  focuses  on  meeting 
the  near-  and  long-term  requirements  of  its  product.  In  fact,  each 
IPT  includes  members  from  the  Sacramento  Air  Logistics  Center, 
who  service  or  update  the  aircraft  when  necessary. 

IPD  in  Action 

Does  IPD  work?  The  Commander  of  Air  Combat  Command 
believes  that  it  does  (and  he  has  been  involved  in  the  program  for 
more  than  10  years).  The  F-22  program  founders,  as  well  as  later 
program  directors,  such  as  Major  General  Robert  Raggio,  also  have 
seen  the  great  payoffs  of  an  integrated,  multifunctional,  empowered 
work  force. 

The  following  memorandum  is  an  actual  IPD.  Written  by  an 
Air  Force  maintenance  officer  assigned  to  the  F-22  program  office, 
it  describes  the  trade-off  process  team  members  used  to  reach  a 
balanced  design  in  the  F-22  lower  tail  section.  In  the  memo,  an 
expert  in  repairing  and  maintaining  the  aircraft  reports  on  reaching 
the  best  overall  solution  to  improve  the  quality  of  the  product  (the 
italics  in  the  memo  are  mine).  Captain  Tinsler’s  example  is  but  one 
of  thousands  of  product-focused  trade-offs  that  occur  everyday  in  the 
F-22  program. 

10  May  94 

Memo  to:  ASC/YFFV 

YFF 

IN  TURN 

Subject:  Trip  Report,  Boeing,  5  May  94 

1.  Purpose  of  visit:  Aft  boom  redesign 

2.  Discussion:  The  objective  of  this  meeting  was  to  reach  agreement  on  the  best 

solution  for  the  aft  boom  stiiShess  design.  Four  designs  were  considered: 

1.  Baseline  design  -  A  large  contoured  access  panel  on  the  bottom  of  the  aft 

boom  with  zero  tolerance  holes,  the  panel  is  considered  part  of  the  stiffness  loop. 

Problems  with  this  approach  are: 

a.  Zero  tolerance  holes  of  0-.003  inches  difficult  to  control  in  manufac¬ 
turing. 

b.  Difficult  to  produce  multi-contoured  stress  panel 


Ptwio  wyrtesy  Loddie^-Marin. 


Captain  Tuisiefs  memo  cwcerning  the  aft  boom  design  was  largely  about  the  frequency  and  ease  of  engine  replacement  and 
nearly  components.  Here,  workers  are  titling  an  F1 1 9  engine  Into  the  aft  boom  area  Tinsler  mentions. 
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c.  Difficult  to  install  the  panel 

d.  Difficult  if  not  impossible  to  drill  new  panel  to  fit  when  replacing  the 
panel 

e.  Requires  load  alleviation  to  install/  remove  the  panel 

2.  Gussets  -  Triangular  shaped  stiffeners  that  tie  frames  1  and  3  to  the  side- 
walls,  panel  is  shaped  same  as  baseline  but  is  not  part  of  stiffness  loop.  The  prob¬ 
lems  referenced  in  Ib-e  above  apply  with  these  additional  concerns: 

a.  Adds  30  pounds 

b.  Installation/removal  of  actuator  support  pins  extremely  difficult 
due  to  gusset  interference 

c.  Increases  number  of  engines  removed  by  39.5  per  squadron  per  year 

3.  Inverted  Actuators  -  This  involves  flipping  the  actuator  servo  valve  at  the 
bottom  of  the  cylinder.  This  design  was  not  seriously  pursued  since  the  actuators 
have  already  passed  CDR  and  this  option  involves  redesigning  the  actuator  (yuch). 

4.  Keelson  Access  -  Bottom  aft  boom  is  permanent  structure  instead  of  an 
access  panel,  access  to  aft  boom  is  through  keelson  panel.  This  is  the  cleanest  solu¬ 
tion  for  the  stiffness  problem  and  saves  significant  weight  but  requires  an  engine 
removal  to  gain  access  to  the  keelson  panel. 

Option  4,  as  originally  proposed,  required  an  engine  removal  to  gain  access  to 
the  horizontal  tail  actuator,  actuator  hydraulic  lines,... fuel  dump  screen. 
Preliminary  numbers  showed  this  design  would  require  an  additional  44  engine 
removals  per  year  per  squadron,  an  idea  the  maintainers  were  adamantly  opposed 
to.  Despite  our  maintainability  objections,  Boeing  vigorously  pursued  the  idea 
since  it  was  a  good  stiffness  solution  with  a  significant  weight  benefit.  Working 
closely  with  the  program  office  and  Air  Combat  Command  maintainers,  Boeing 
looked  at  how  best  to  address  our  concerns  and  presented  their  recommended  solu¬ 
tion  at  this  meeting.  Their  recommendation  was  a  modified  option  4  approach.  The 
highlights  of  this  design  are: 

a.  Small  access  panel  on  bottom  of  aft  boom  for  fuel  pump  removal  and 
limited  hydraulic  line  B-nut  access,  the  rest  of  the  skin  is  perma¬ 
nent 

b.  Two  keelson  access  panels  behind  the  AM&S  for  access  to  the  fuel 
dump  valve  and  screen  and  access  to  the  horizontal  tail  mount  bolts: 
access  to  these  panels  gained  by  removing  the  AM&S 

c.  Large  keelson  access  panel  for  actuator  removal;  access  to  this  panel 
requires  engine  removal. 

This  modified  option  4  reduces  engine  removals  to  5  per  year  compared  to  44, 
significantly  improves  access  to  the  hydraulic  lines  on  top  of  the  actuator,  saves  the 
aircraft  about  45  pounds,  and  reduces  overall  access  time  (based  on  compsuing  90 
minute  engine  remove  and  replace  to  load  alleviation  jacking  and  removal  of  coating 
and  300-1-  fasteners).  Removing  and  installing  the  actuator  is  the  only  reason  to 
remove  the  engine,  a  tradeoff  the  program  office  and  Air  Combat  Command  main- 
tainers  agreed  to  based  on  the  benefits  listed  above. 

Conclusions:  When  Boeing  originally  proposed  this  idea  to  the  maintainers,  we 
told  them  we  considered  pulling  an  engine  to  gain  actuator  access  a  non-option. 
They  listened  to  our  concerns  and  in  a  two-week  span  came  up  with  the  modified 
option  solution.  To  further  convince  us,  they  mocked  up  the  aft  boom  complete  with 
actuator,  fuel  pump,  fuel  dump  valve  and  all  associated  tubing.  Fundamentally  we 
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still  dislike  the  idea  of  removing  the  engine  to  gain  access  to  other  components. 
However,  I  believe  we  made  the  right  tradeoffs  and  came  up  with  the  best  solution  for 
the  airplane. 

James  A.  Tinsler,  Capt,  USAF 
Vehicle  Management  System  Maintainer 

A  Final  Note  on  IPTs 

The  F-22  program  IPTs  are  successful  at  developing  balanced 
solutions  that  have  the  commitment  of  the  stakeholders  who  must 
carry  them  out.  These  Govemment/contractor  teams  fully  engage 
the  ACC  customer-user.  The  user,  being  both  part  of  the  design  cycle 
and  the  most  important  part  of  the  decision  process,  fully  sup¬ 
ports  the  decision  reached  by  the  team.  IPD  is  instrumental  in 
user  satisfaction. 

It  is  perhaps  obvious  that,  because  IPTs  implement  IPD,  the  char¬ 
acteristics  of  a  successful  IPT  echo  the  elements  of  IPD  in  Figure  7-2. 
However,  as  shown  in  Figure  7-7,  there  is  an  additional  characteristic 
necessary  for  a  successful  IPT.  Each  team  must  know  the  limits  of  its 
decisionmaking  authority,  or  fenceposts  (as  described  in  chapter  4). 
When  the  team  understands  what  the  constraints  of  cost,  schedule, 
and  performance  are  for  its  product  and  what  decisions  must  be  coor¬ 
dinated  at  higher  (or  lateral,  or  sometimes,  lower)  levels,  it  is  fully 
empowered  to  manage  its  portion  of  the  program. 

Results 

Multidisciplinary  teams,  focused  on  meeting  the  short-  and  long¬ 
term  needs  of  the  customer-user,  are  a  powerful  tool  in  solving  com¬ 
plex  problems.  To  reap  the  full  benefit  of  integrated,  product-focxised 
teams,  the  teams  need  a  tool  set  that  is  also  integrated.  Chapter  8 
describes  the  tools  that  support  the  F-22  program  IPD. 
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Figure  7-7.  Characteristics  of  Successful  IPTs 


Successful  IPTs 

1 .  Are  customer-focused 

2.  Are  product-oriented 

(Where  a  “product"  can  be  an  item,  a  process  or  an  issue) 

3.  Are  empowered,  cross-functional  teams,  including 

•  All  stakeholders  involved 

•  All  necessary  disciplines/organizations  (including 
the  customer) 

4.  Take  a  long-term  view 

5.  Understand  the  limits  of  their  decisionmaking  authority 
(fenceposts) 
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Everyone  is  lined  up.  They  all  know  where  they  are  within  the 
contract,  know  what  specs  they  are  working  to,  what  part  they 
are  working  to,  how  many  resources  they  have,  and  how  much 
time  they  have  to  do  it 


Colonel  Wallace  T.  Bucher, 
at  the  start  of  the  engineering  and 
manufacturing  development  phase 


Background 

The  first  half  of  the  F-22  management  system,  integrated  prod¬ 
uct  development  (IPD,  described  in  chapter  7)  is  supported  by 
the  second  half,  the  management  tools  used  by  all  levels  of  the 
program  (see  Figure  8-1).  Everyone  from  the  Program  Director  to  the 
individual  members  of  the  integrated  product  teams  (IPTs)  use  this 
integrated  management  framework  for  the  day-to-day  execution  of  the 
F-22  development  effort. 

The  purpose  of  the  demonstration/validation  phase  of  the  ATF 
program  was  to  reduce  risk.  In  addition  to  addressing  the  risks  asso¬ 
ciated  with  requirements  and  technology,  the  ATF  program  office  also 
actively  worked  to  reduce  risks  by  controlling  costs  and  schedules,  and 
by  meeting  performance  expectations  for  the  development  phase. 
Early  in  demonstration/validation,  the  program  office  established  an 
Acquisition  Strategy  Working  Group  to  address  the  business  aspects 
of  the  program,  including  the  acquisition  strategy  and  management 
framework  for  the  engineering  and  manufacturing  development 
(EMD)  phase.  This  working  group,  led  by  the  Deputy  Program 
Director,  Colonel  Wallace  T.  Bucher,  included  finance  and  contracting 
personnel,  engineers,  managers,  logistics  experts,  and  test  personnel 
from  the  program  office  and  the  contractors.  They  discussed  such 
items  as  what  t3rpes  of  contracts  to  award,  how  to  motivate  the  con¬ 
tractors  to  perform,  what  type  of  data  they  would  need,  and  how  to 
track  progress — every  aspect  of  the  program  that  was  not  technical. 

The  Acquisition  Strategy  Working  Group  had  two  primary  objec¬ 
tives  in  developing  the  new  management  structure:  first,  plan  and 
commit  to  an  executable  program — ^track  and  manage  to  their  plan — 
and,  second,  decide  how  to  evaluate  and  reward  contractor  performance. 
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Figure  8*1.  Elements  of  Risk  Management:  The  Management  System 


The  working  group  prototyped  and  tested  several  techniques  during  the 
engine's  second-phase  demonstration/validation  contracts.  Armed  with 
the  lessons  learned  from  the  engine  experience,  the  ATF  program 
refined  the  concepts  and  created  an  integrated  management  framework. 

The  Integrated  Management  Framework 

The  F-22  integrated  management  framework,  shown  in  Figure 
8-2,  can  be  divided  into  three  categories  of  management  tools.  The 
first  category,  planning  tools,  includes  the  requirements  called  for  in 
the  functional  system  specifications:  the  work  breakdown  structure, 
the  statement  of  work,  and  the  integrated  master  plan.  The  second 
category  of  tools  focuses  on  tracking:  the  integrated  master  sched¬ 
ule,  technical  performance  measures,  cost/schedule  control  systems 
criteria  and  other  financial  tools,  and  closure  plans.  The  final  cate¬ 
gory  includes  the  means  of  giving  feedback  to  the  team  on  perform¬ 
ance,  delivered  through  the  award  fee  process. 

Requirements 

As  discussed  in  chapter  6,  the  ATF  program  documented  the 
requirements  for  the  weapon  system  in  the  preUminary  system  speci¬ 
fication.  The  bidding  contractors  submitted  their  version  of  the  weapon 
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system  specification  as  part  of  their  proposal  for  the  EMD  phase.  The 
contractor  chosen  for  EMD  then  allocated  the  top-level  requirements 
from  the  weapon  system  specification  to  more  detailed  specifications 
for  the  major  portions  of  the  weapon  system.  These  functional,  “T^pe 
A’’  specifications  covered  15  separate  areas,  including  the  weapon 
system,  the  air  vehicle,  the  support  system,  the  training  system,  and 
other  major  portions  of  the  air  vehicle  (such  as  avionics).  The  engine 
contract  includes  three  additional  functional  specifications  for  the 
engine  and  the  engine's  support  and  training  systems. 

Work  Breakdown  Structure 

The  work  breakdown  structure  serves  as  a  program  outline.  It 
graphically  depicts  the  way  in  which  the  contractor  intends  to  per¬ 
form  the  work.  As  such,  the  F-22  (and  the  F119)  work  breakdown 
structure  is  tailored  to  the  development  and  manufacturing  process. 

The  work  breakdown  structure  serves  as  the  basis  for  the  F-22 
single-numbering  system  and  is  needed  to  ensure  an  integrated  set  of 
management  tools.  All  specifications,  as  well  as  the  statement  of 

Figure  8-2.  The  F-22  Program’s  Integrated  Management  Framework 

I - L 
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work,  integrated  master  plan  and  schedule,  technical  performance 
measures,  and  cost  reporting,  are  arranged  according  to  the  work 
breakdown  structxzre.  Because  it  is  organized  primarily  by  products 
and  their  constituent  subproducts,  and  because  it  is  the  main  thread 
that  runs  throughout  the  management  system,  it  reinforces  team 
focus  on  the  product.  As  a  result,  IPTs  are  also  organized  according  to 
the  work  breakdown  structure. 

Statement  of  Work 

The  statement  of  work  describes  the  minimal  essential  require¬ 
ments  of  the  program — that  is,  what  the  contractor  must  do.  The 
following  paragraph  is  an  excerpt  from  the  statement  of  work  for  the 
vehicle  management  system  (VMS). 

1500  VEHICLE  MANAGEMENT  SYSTEM 

The  contractor  shall  design,  develop,  integrate,  test,  qual¬ 
ify  and  prepare  for  production  of  a  VMS  that  meets  the 
Weapon  System  Specification,  in  accordance  with  the 
activities  described  in  the  IMP.  The  contractor  shall 
analyze,  verify,  and  document  the  design  of  each  VMS 
configuration  item  and  integrate  them  within  the  VMS. 

The  contractor  shall  integrate  the  VMS  with  other  ATF 
subsystem  elements,  the  air  vehicle,  and  the  support  and 
training  systems.  The  contractor  shall  develop  the 
requirements  for,  coordinate  the  use  of,  and  control 
configuration  of  an  integrated  VMS  test  facility  for  the 
purpose  of  verifying  and  validating  certain  VMS  require¬ 
ments  and  the  integration  of  the  VMS  with  other  sub¬ 
system  elements. 

The  F-22  statement  of  work  is  different  from  those  in  earlier 
programs  in  that  the  contractor  wrote  it.  In  typical  acquisitions  the 
Government  presents  the  contractor  with  a  statement  of  work  and  a 
detailed  specification,  but  for  the  ATF  program  the  Government  only 
supplied  sample  language  for  its  minimum  essential  activities  (like 
the  Government  test  program).  The  ATF  program  leadership  imder- 
stood  that  the  contractors  knew  how  to  develop  a  fighter  much  bet¬ 
ter  than  the  Air  Force  did.  Fain  explained  it  this  way: 

When  you  go  out  and  build  a  house,  you  don’t  schedule  the 
jobs  and  tell  the  general  contractor  what  they’re  to  do.  You 
tell  them  to  build  you  a  house.  The  general  contractor 
figures  out  how  to  build  it — it’s  their  job.  They  are  the 
expert!  To  have  the  Government  come  in  and  tell  the 
contractor  how  to  build  a  weapon  system  by  giving  them  a 
statement  of  work  is  just  plain  stupid! 
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Integrated  Master  Plan 

The  integrated  master  plan  (IMP)  is  an  event-driven  planning 
document  prepared  by  the  contractor  and  used  by  the  IPTs  to  manage 
the  development  effort. 

Birth  of  the  integrated  master  plan  and  integrated  master 
schedule.  Marcia  Irvin  of  the  F-22  program  office  described  the 
evolution  of  the  IMP  and  IMS  as  follows: 

Early  in  the  process,  the  engineering  commimity  within  the 
SPO  [System  Program  Office]  encoxiraged  the  use  of  a 
Systems  Engineering  Master  Schedule  (SEMS),  The  belief 
was  that  the  contractor  would  surely  need  to  use  this  or 
some  similar  planning  tool  to  accomplish  a  program  of  this 
size.  Hence,  SEMS  became  the  laimching  point  for  schedule. 

One  problem  noted  by  the  Acquisition  Strategy  Working  Group, 
however,  was  that  the  SEMS  included  dates,  and,  as  a  contractual 
annex,  would  require  contract  modification  each  time  the  program 
schedule  changed.  The  proposed  solution  was  to  create  a  solely 
event-based  (no  dates)  planning  document  that  could  be  incorpo¬ 
rated  into  the  contract  and  maintain  a  data  item  to  provide  the 
supporting  schedules  for  the  contractual  plan.  So,  SEMS  was  refor¬ 
matted  somewhat  and  the  dates  were  removed  to  create  the  event- 
based  contractual  plan.  The  supporting  data  item  developed  was 
known  as  the  Systems  Engineering  Detailed  Schedule  (SEDS). 

The  one  fallacy  with  SEMS  and  SEDS  was  that  while  they  would 
easily  have  satisfied  the  engineering  community,  management 
would  not  have  had  any  insight  into  functional  processes  necessary 
for  program  accomplishment.  The  intermediate  solution  was  a 
“sister”  set  of  schedules  to  SEMS  and  SEDS  that  would  be  process 
oriented,  the  Management  Master  Plan  (MMP)  and  the  Manage¬ 
ment  Master  Plan  Schedule  (MMPS).  Trial  application  of  this  con¬ 
cept  on  the  ATF  engine  second  demonstration/validation  contract, 
however,  surfaced  another  problem.  The  gray  area  between  the 
technical  focus  of  SEMS/SEDS  and  management  (process)  focus  of 
MMP/MMPS  created  an  integration  problem.  Specificedly,  some 
items  would  either  be  duplicated  on  both  sets  of  documents  or 
they  might  not  be  included  at  all  because  it  wasn’t  clear  which  set  of 
documents  was  appropriate.  Ultimately,  the  solution  was  to  elimi¬ 
nate  the  gray  area.  SEMS  and  MMP  were  merged  to  become  the 
Integrated  Master  Plan.  The  SEDS  and  MMPS  were  merged  to 
become  the  Integrated  Master  Schedule. 

As  just  described,  the  major  difference  between  previous  tools, 
such  as  the  system  engineering  master  schedule,  and  the  IMP  is  that 
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the  IMP  contains  all  program  activity  (not  just  engineering)  and 
does  not  contain  calendar  dates. 

The  IMP  is  part  of  the  contract  so  that  no  one  can  change  the 
program  activities  without  agreement  between  the  contractor  and 
the  Grovemment.  It  expands  on  the  tasks  in  the  statement  of  work 
and  defines  the  milestones  for  both  the  products  and  the  processes 
needed  to  design,  test,  and  produce  the  F-22  weapon  system.  As 
described  in  the  ATP  request  for  proposal,  the  IMP  is  defined  by  four 
elements — events,  significant  accomplishments,  accomplishment 
criteria,  and  detailed  tasks — that  describe  the  activity  and  define  its 
successful  completion.  Their  specific  definitions  follow: 

•  Event:  The  conclusion  or  initiation  of  an  interval  of  major 
program  activity 

•  Significant  accomplishment:  A  desired  result  at  a  specified  event 
that  indicates  a  level  of  design  maturity  (or  progress)  directly 
related  to  each  product  or  process 

•  Accomplishment  criterion:  A  definitive  measure  or  indicator  that 
the  level  of  maturity  (or  progress)  has  been  achieved 

•  Detailed  tasks:  Detailed  work  to  be  completed  in  support  of  a 
specific  significant  accomplishment. 

Figure  8-3  shows  a  portion  of  the  work  breakdown  structure; 
Item  1500  of  the  IMP  for  the  VMS.  Note  that  the  event  is  critical 
design  review  and  that  the  significant  accomplishment  is  completion 
of  detailed  design.  Also  note  the  single-numbering  reference  to  the 
statement  of  work  sections. 

Because  of  the  details  provided  by  the  contractors  in  the  IMP, 
the  program  office  prohibited  use  of  all  standard  “specialty”  plans 
(such  as  quality  plans  or  configuration  management  plans). 
Presenting  these  specialty  details  in  the  IMP  gave  the  teams  a 
better  appreciation  for  how  specialty  areas  had  to  work  with  other 
parts  of  the  program  and  has  greatly  helped  reinforce  integration. 

Integrated  Master  Schedule 

The  IMS  expands  on  the  IMP  by  providing  the  tasking  and 
timing  of  the  work  effort  required  to  support  the  IMP  events.  The 
IMS  is  not  part  of  the  contract,  which  allows  the  IPTs  the  flexibility 
to  manage  their  program  without  constantly  changing  the  contract. 
The  IMS  lays  the  foundation  for  budget  and  schedule  planning 
and  links  directly  to  the  reporting  of  cost  and  schedule.  Figure  8-4 
shows  the  IMS  for  the  VMS. 
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Figure  8-3.  Vehicle  Management  System  Integrated  Master  Plan 


EVENT 

Significant  Accomplish 

Accomplishment  Criteria 

ACTIVITY 

SOW 

Tasks 

NUMBER 

REF 

VMS  CRITICAL  DESIGN  REVIEW  (CDR) 

J03 

1500 

VMS  DETAILED  DESIGN  COMPLETE 

J0306 

1500 

VMS  Computer  Resources  CDR  complete  for  each  Cl 

J030616 

1500 

MEDS  Design  review  update  (DRU)  complete 

J030616A 

1520 

PICC  CDR  complete 

J030616B 

1520 

Power  supply  CDR  complete 

J030616C 

1520 

METS  CDR  complete 

J030616D 

1520 

TPS  CDR  complete 

J030616E 

1520 

TPS  SW  CDR  complete 

J030616F 

1520 

VMS  VKS  CDR  complete 

J030616G 

1520 

METS  software  baseline  released 

J030616H 

1520 

VKS  CDR  update  complete 

J030616I 

1520 

PICC  Spec  V&V  plan  complete  and  available 

J030616J 

1520 

VMS  Actuators  CDR  Complete  for  each  Cl 

J030617 

1530 

VMS  Actuator  CDR  complete  for  each  Cl 

J0306171 

1530 

VMS  Actuator  design  and  analysis 

J0306172 

1530 

VMS  Actuator  modeling  analysis  complete 

J0306173 

1530 

VMS  Actuator  vendor  test  procedures  complete 

J0306174 

1530 

Technical  Performance  Measures 

Metrics  motivate  behavior.  However,  you  must  be  very, 
very  careful  as  to  what  you  measure. 

Major  General  Robert  Raggio 

Why  did  I  invent  TPMs?  Because  I  wanted  to  put  disci¬ 
pline  into  the  technical  aspects  of  the  program. 

Eric  E.  Abell 

Technical  performance  measures  (TPMs)  are  a  tool  to  track  tech¬ 
nical  progress  toward  meeting  the  weapon  system  specifications. 
Abell,  knew  that  in  past  programs  the  cost  and  schedule  elements  of 
risk  could  be  tracked  with  metrics  (such  as  the  cost/schedule  control 
system,  or  critical  path  charts  to  manage  the  schedule).  He  wanted 
a  similar  tool  to  allow  him  to  track  technical  risk. 

The  F-22  program  uses  TPMs  at  all  levels  to  display  results- 
to-date  along  with  projected  results,  to  show  the  technical  maturity 
of  each  product,  and  to  present  trends  as  well  as  projected  perform¬ 
ance  that  the  IPTs  expect  their  product  to  achieve.  An  important 
benefit  of  TPMs  is  that  they  provide  an  historical  record  of  a  prod¬ 
uct's  performance. 


Figure  8-4.  Vehicle  Management  System  Integrated  Master  Schedule 


Activity  Name 

Activity 

Code 

VMS  Critical  Design 

J03 

Review 

VMS  Detailed  Design 

J0306 

Complete 

1993 


Figure  8-5  shows  the  standard  TPM  chart  format.  On  the  top  of 
TPMs  are  the  program  milestones:  requirements  design  review 
update  (RDRU),  preliminary  design  review  (PDR),  and  critical  design 
review  (CDR),  The  information  box  in  the  upper-right  corner  shows 
the  work  breakdown  structure  reference,  the  TPM  source  for  require¬ 
ments  (usually  the  specification),  the  contractor  and  Government 
owners  of  the  product  measured,  and  the  date  of  the  information.  The 
graph  plots  performauice  against  time.  For  classified  TPMs  the  values 
are  normalized  to  show  differences  from  the  classified  requirement 
(usually  making  the  TPM  unclassified). 

The  target  value,  shown  by  the  dashed  line  in  the  center,  shows 
the  specification  value  for  the  product.  The  round  dot  identifies  the 
proven  performance  value  as  derived  by  development  accomplish¬ 
ments  by  a  certain  date.  The  achieved-to-date  value,  based  on  initial 
lab  tests  or  tests  coupled  with  calculations,  typically  lies  below  the 
target  line  and  increases  as  development  progresses  and  the  demon¬ 
strated  performance  of  the  product  improves.  Thus,  over  time,  the 
value  of  the  current  update  should  approach  the  target  value. 

The  current  estimate,  shown  by  a  solid  triangle,  is  the  most 


Figure  8-5.  Technical  Performance  Measure  Chart  Format 
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recent  forecast  from  the  IPT  on  how  it  believes  the  product  will  per¬ 
form  when  first  produced.  When  this  value  strays  below  the  target, 
the  IPT  (and  its  higher  level  IPTs)  know  to  start  corrective  action. 

Like  any  projection,  the  TPM  chart  includes  assumptions.  The 
confidence  band  shows  the  possible  range  of  the  current  estimate  for 
the  best-  and  worst-case  assumptions.  In  this  manner,  the  reader  of 
the  chart  understands  the  magnitude  of  the  accuracy  and  maturity  of 
the  current  estimate  and  the  range  of  values  for  final  performance. 

The  activity  bar  at  the  bottom  of  the  graph  identifies  when  the 
IPT  generated  new  data  for  the  TPM.  An  attached  sheet  (called  the 
TPM  summary)  describes  the  estimates,  the  reasons  values  have 
changed  from  earlier  information,  the  TPM  measures,  and  any 
assumptions  or  additional  information  about  the  TPM. 

The  F-22  TPMs  are  tiered  by  product.  Virtually  every  TPM  on 
each  product  describes  performance  that  supports  the  performance 
of  a  higher  level  item,  which  supports  that  of  a  higher  level  item, 
ultimately  contributing  to  the  overall  performance  of  the  aircraft, 
engine,  or  support  or  training  system.  The  highest  level  TPMs  track 
the  F-22  critical  characteristics  (shown  in  Figure  8-6).  By  DOD 
regulation,  failure  to  meet  any  of  the  critical  characteristics  allows 
the  Secretary  of  Defense  to  review  the  program  and  assess  whether 
it  should  continue.  The  F-22  Program  Director  reviews  the  TPMs 
relevant  to  each  critical  characteristic  weekly. 

Figure  8-7  shows  the  independent  airlift  TPM.  The  program 
office,  the  contractor,  and  the  ACC  members  of  the  Support  System 
IPT  use  this  TPM  to  monitor  how  many  C-141B  equivalent  loads 
they  would  need  to  deploy  and  maintain  a  squadron  of  24  F-22s 
for  30  days.  The  requirement  is  to  deploy  with  no  more  than  eight 
C-141Bs,  half  of  what  is  needed  today  for  the  F-15.  This  TPM  is  a 
good  example  of  the  power  of  metrics.  Whenever  the  current  esti¬ 
mate  approaches  the  target,  a  great  deal  of  management  attention 
goes  into  finding  out  what  has  changed  and  what  the  IPT  will  do  to 
lower  the  C-141  forecast. 


Figure  8-6.  F-22  Critical  Characteristics 
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The  forward  fuselage  of  the  first  F-22  is  lowered  into  the  fuselage  mate  tool.  Later  the  mid  and  aft  fuselage  components  will  be 
added  to  this  tool. 


Cost/Schedule  Control  Systems  Criteria 

The  F-22  program  monitors  cost  performance  in  three  ways: 
through  the  cost  performance  report,  the  overhead  cost  report,  and 
the  design-to-cost  status  report. 

Cost  performance  report.  The  primary  method  of  cost  tracking 
comes  from  the  contractor’s  internal  cost  control  system  as  validated 
by  the  Government  to  meet  defined  cost/schedule  control  systems  cri¬ 
teria.  Each  month,  the  contractors  generate  a  cost  performance 
report  that  shows  their  expenditures  to  date  compared  with  their 
planned  expenditures  and,  thus,  their  progress  in  terms  of  cost  and 
schedule  variance.  The  F-22  program  office  worked  with  both 
Lockheed  and  Pratt  &  Whitney  to  tailor  a  portion  of  the  report  to 
deliver  a  Format  2B,  which  shows  the  cost  status  by  IPT.  This  infor¬ 
mation  is  extremely  valuable  to  the  IPT^  in  monitoring  whether  they 
are  performing  according  to  plan. 

The  Program  Director  (and  the  Government/contractor  IPTs) 
receives  a  preliminary  cost  performance  report,  called  a  flash  report. 


Figure  8-7.  Independent  Airlift  Technical  Performance  Measure 
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no  later  than  10  days  after  the  contractor  accounting  month  ends. 
This  report  is  unaudited  but  serves  to  give  the  IPTs  and  the  Govern¬ 
ment  and  contractor  Program  Directors  early  insight  into  cost  and 
schedule  issues.  Both  contractors  deliver  the  final  cost  performance 
report  30  days  after  the  close  of  the  accounting  month.  In  earlier 
programs,  the  cost  performance  report  typically  would  not  arrive 
until  up  to  90  days  after  the  close  of  the  accounting  month — making 
the  data  very  old  and  less  useful.  The  program  office  insisted  that 
the  contractors  deliver  the  cost  performance  report  as  soon  as  possi¬ 
ble.  It's  important  for  future  Program  Directors  to  note  that  both 
contractors  initially  believed  that  a  10-day  delivery  of  initial  cost 
performance  data  was  impossible. 

Overhead  cost  report.  During  the  source  selection  for  EMD, 
the  Secretary  of  the  Air  Force,  Donald  B.  Rice,  expressed  concern 
about  the  potential  for  increased  overhead  rates  for  each  of  the 
contractors.  With  the  reduction  of  defense  contracts,  the  contractors 
would  need  to  spread  their  fixed  costs  over  a  smaller  number  of 
ongoing  contracts  (such  as  the  ATF  weapon  system  and  the  ATF 
engine),  directly  increasing  the  costs  allocated  to  the  remaining 
contracts.  Secretary  Rice  feared  that  the  cost  of  the  ATF  program 
would  increase  greatly  as  a  result  of  the  projected  increase  in  over¬ 
head  rates.  He  directed  the  F-22  program  to  create  a  method  of 
tracking  overhead  costs  that  would  show  clearly  what  he  concluded 
would  be  the  major  financial  challenge  for  the  program. 

Following  their  selection  as  F-22  contractors,  Lockheed  and  Pratt 
&  Whitney  worked  with  the  program  office  to  establish  a  method 
of  tracking  and  reporting  overhead  costs — called,  cleverly  enough, 
the  overhead  cost  report.  The  contractors  initially  defined  their 
overhead  rates  at  the  start  of  the  EMD  contract.  Tbday,  they  submit 
the  report  only  when  their  overhead  rates  change  from  this  initial 
baseline  to  document  what  changed  and  why.  The  Government  and 
contractors  then  together  define  actions  to  control  and  reduce  the 
growth  in  overhead  costs. 

Design-to-cost  status  report.  The  founding  fathers  wanted  all 
IPTs  to  focus  on  a  life-cycle  perspective.  To  make  decisions  properly 
the  IPTs  need  to  know  the  cost  of  the  decisions,  not  just  for  the 
immediate  future  (the  development  cost),  but  also  for  downstream 
production.  Through  the  use  of  design  to  cost,  the  IPTs  establish 
production  cost  as  another  performance  variable.  The  monthly 
design-to-cost  status  report  allocates  production  cost  targets  and 
identifies  critical  areas  and  problems  that  may  cause  the  IPTs  to 
exceed  their  cost  goals.  The  IPTs  also  prepare  a  TPM  that  tracks 
their  product's  projected  production  cost.  The  regular  reporting  of 
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this  TPM  gives  the  IPTs  a  clear  way  to  document  the  actions  they’ve 
taken  to  reduce  production  costs. 

Closure  Plans 

The  fourth  IPT  tool  for  tracking  progress  and  managing  their 
program  is  the  closure  plan.  The  F-22  program  current  IMS  contains 
over  30,000  items.  As  detailed  as  that  may  at  first  appear,  sometimes 
it  requires  judgment  to  determine  when  an  IPT  can  declare  an  item 
complete.  The  closure  plan  is  simply  a  formal  agreement  between  the 
contractor  and  the  Government  specifying  actions  that  will  occur 
before  completion  of  an  item.  The  Government  and  contractor  team 
members  jointly  prepare  and  sign  this  plan  to  define  their  course  of 
action.  Closure  plans  give  the  IPTs  the  flexibility  to  manage  their 
portions  of  the  program  within  a  formalized  agreement. 

Award  Fee 

At  the  end  of  the  demonstration/validation  phase,  the  program 
office  (and  the  bidding  contractors)  wanted  to  ensure  that  the  devel¬ 
opment  contracts  would  be  fair  to  both  the  Government  and  the 
contractors.  The  Government  thus  decided  to  use  cost-plus-award- 
fee  contracts  in  the  F-22  and  F119  development  efforts.  With  this 
type  of  contract,  the  Government  agrees  to  pay  the  contractors  all  of 
their  allowable  costs  and  to  pay  an  award  relative  to  how  well  they 
meet  their  cost,  schedule,  and  performance  requirements.  Both  the 
F-22  and  F119  contracts  specify  a  4  percent  fixed  fee  (to  cover  the 
contractor’s  unallowable  but  required  costs)  and  a  9  percent  award 
fee  (a  percentage  of  the  contract  value).  The  award  fee  establishes  a 
pool  of  money  available  for  the  contractor  to  earn  based  on  perform¬ 
ance.  As  Raggio  describes  it,  this  contract  allows  an  equal  sharing  of 
risk.  The  Government  agrees  to  cover  the  costs  to  develop  the  F-22 
(or  F119),  and  the  contractor  agrees  to  put  its  entire  award  profits 
(the  9  percent)  on  the  line. 

The  award  fee  is  the  primary  (some  say  only)  tool  for  motivating 
contractor  performance.  Meeting  contractual  requirements  (including 
those  for  cost,  schedule,  and  technical  performance)  results  in  an 
award  of  100  percent  of  the  award  fee  pool.  The  Gk)vemment  deter¬ 
mines,  and  pays,  the  award  fee  every  6  months.  Monthly,  the  IPTs 
record  contractor  performance,  citing  both  strengths  and  weakness¬ 
es,  and  the  IPT  leaders  pass  the  assessment  directly  to  their  contrac¬ 
tor  counterparts  as  feedback.  Three  months  and  6  months  into  every 
period,  the  contractors  and  the  Government  award  fee  board  meet  to 
exchange  views  on  contractor  performance.  In  this  way,  the  contractor 
gets  regular  information  on  how  it  is  doing  and  where  the 


_ Integrated  Tool  Set  1 07 

Government  believes  the  contractor  needs  to  improve.  The  objective  of 
the  Government  is  very  clear:  for  the  contractor  to  earn  100  percent  of 
its  profits,  it  must  fully  meet  its  cost,  schedule,  and  performance 
reqxiirements.  It  is  important  to  note  that  the  program  office  accom¬ 
plishes  award  fee  tasks  on  a  daily  basis  using  standard  information 
(nothing  is  tracked  just  for  award  fee  purposes). 

A  significant  feature  of  the  F-22  award  fee  process  is  that  the 
program  director  determines  the  fee  for  the  period,  in  contrast  to  other 
programs,  in  which  the  fee-determining  official  typically  serves  at  a 
level  above  the  program  director.  The  conventional  logic  argued  that  the 
fee  official  needed  to  be  somewhat  removed  from  the  program  to  ensure 
an  objective  decision.  Diiring  the  F-22  EMD  Defense  Acquisition  Board 
review,  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition,  John  J. 
Welch,  argued  strongly  that  the  program  director  should  be  allowed 
to  determine  the  award  fee  because  it  is  the  primary  motivator  for  the 
contractor.  To  take  the  award  fee  decision  away  from  the  program  direc¬ 
tor  leaves  the  the  director  with  little  influence  over  the  contractor,  for 
whoever  decided  the  award  fee  amount  woxild,  in  effect,  be  the  real 
director.  The  fact  that  fi-om  1991  to  1995  the  contractors  never  protest¬ 
ed  the  award  shows  that  a  program  director  can  fairly  determine  it. 

The  program  office  designed  the  award  fee  process  to  be  a  win-win 
relationship,  a  vital  part  of  the  integrated  management  framework. 
The  integrated  tool  set  helps  evaluate  contractor  performance  objec¬ 
tively,  and  the  award  fee  program  promotes  frequent  communication, 
early  problem  identification  and  resolution,  proactive  management, 
and  the  teamwork  needed  to  develop  the  weapon  system  and  engine. 

The  IPT  Management  Tool  Set 

Every  design  engineer,  program  manager,  logistics  expert,  main¬ 
tenance  expert,  manufacturing  engineer,  and  finance  or  contracting 
specialist  serving  on  an  IPT  has  a  set  of  IPT  management  tools.  The 
tool  set  begins  with  the  identification  of  a  work  breakdown  structure 
element  (such  as  1500  for  the  VMS),  which  has  its  own  set  of  tools 
(Figure  8-8).  The  IPT  member  uses  the  appropriate  statement-of- 
work  tasks  and  IMP  sections  to  define  the  work,  the  appropriate 
specification  paragraph  to  define  the  requirements,  and  the  budget, 
as  allocated  by  the  contractor  team  members,  to  define  the  resources 
available  for  the  job. 

The  IPT  tracks  its  performance  against  the  IMS  for  the  task,  mon¬ 
itors  and  tracks  the  technical  performance  of  its  product  through  the 
correct  TPMs,  and  appraises  its  cost  performance  by  analyzing  the 
Format  2b  in  the  cost  performance  report.  Finally,  the  contractor 
members  of  the  IPT  receive  feedback  through  the  award  fee  process. 
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Figure  8-8.  IPT  Management  Tool  Set 

The  IPT  tool  set  includes: 

Planning  and  Commitment  Tools 

•  Task  from  the  statement  of  work 

•  Integrated  master  plan  section 

•  Specification  paragraph 

•  Budget 

Tracking  Tools 

•  Integrated  master  schedule  section 

•  Technical  performance  measures 

•  Cost  performance  reports 

Motivating  Tools 

•  Award  fee 


Using  the  IPT  Tool  Set 

Figure  8-9  gives  a  graphic  representation  of  how  the  IPTs  use 
the  tool  set  to  track  and  manage  the  development  of  their  product. 

Using  the  work  breakdown  structure  and  the  IMS  for  their 
product,  the  contractor  members  of  the  IPTs  build  the  work  package 
and  allocate  funds  to  the  task.  The  teams  capture  their  progress 
through  the  cost  performance  report  and  the  TPMs  in  the  contrac¬ 
tor’s  data  base  (to  which  the  Government  program  office  has  full 
access).  The  IPTs  then  record  their  development  status  on  the  flash 
reports  and  subsequent  cost  performance  reports,  which  show  the 
team’s  cost  and  schedule  variances;  document  progress  against  the 
schedule  through  regular  reporting  of  IMS  activities;  and  describe 
technical  progress  according  to  the  technical  performance  measures 
for  their  product.  In  addition  to  day-to-day  conversations,  the  con¬ 
tractor  receives  feedback  through  the  award  fee  process. 

Summary 

The  ATF  foimders  designed  the  integrated  tool  set  to  support  their 
objectives  of  integration,  focus  on  the  product,  and  early  up-front  plan¬ 
ning.  The  principles  of  successful  program  management  (shown  in 
chapter  3)  indicate  that  the  integrated  tool  set  supports  them.  IPD  is  a 
powerful  concept  but  the  team  has  to  have  the  right  tools.  The  F-22 
integrated  management  framework  provides  the  tools  and  empowers 
IPTs  to  successfully  develop  the  F-22  fighter  and  F119  engine. 


Figure  8-9.  Using  the  Integrated  Tool  Set  to  Track  a  Product 
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The  F-22  program  has  been  deliberately  paeed.  Its  foundations, 
including  the  competitive  prototyping  and  risk  reduction,  are 
admirable.  It  is  reaching  the  development  milestone  with  a 
management  pedigree  for  success. 

Aviation  Week  and  Space  Technology 
(April  29,  1991,  Page  68) 


The  F-22  acquisition  effort  entered  the  engineering  and  manu¬ 
facturing  development  (EMD)  phase  in  1992.  How  well  are 
these  tools  and  methods  working  today?  Is  the  F-22  program 
really  a  model  acquisition  program,  as  former  Secretary  of  Defense 
Richard  Cheney,  Under  Secretary  of  Defense  John  M.  Deutch,  Senator 
Sam  Nunn,  and  others  claim?  To  answer  the  question  this  chapter 
includes  the  views  of  the  program  customers — Air  Combat  Command 
(ACC),  the  operational  user,  for  whom  the  program  office  is  designing 
the  weapon  system,  as  well  as  the  leadership  of  Air  Force  Acquisition. 
The  chapter  also  includes  the  perspective  of  the  program  office  after  4 
years  experience  with  integrated  product  development  (IPD)  and  the 
integrated  management  system.  The  chapter  concludes  with  a  review 
of  the  advantages  of  the  F-22  program  structure  and  the  lessons 
learned  from  the  F-22  program  as  of  mid-1996. 

The  Operator’s  View 

The  warfighting  customer  perspective  of  the  F-22  program  can  be 
assessed  through  the  views  of  General  John  Michael  Loh,  Commander 
of  Air  Combat  Command  from  1991  until  his  retirement  in  1995.  Loh 
was  closely,  and  continuously,  involved  with  the  ATF  program  since 
the  early  1980s  (he  wrote  the  first  ATF  requirements  document).  As 
the  ATF  program  progressed  through  the  demonstration/validation 
phase  and  its  subsequent  requirements  refinement,  he  served  as  the 
Air  Force  Director  of  Operational  Requirements  when  that  responsi¬ 
bility  resided  with  the  Deputy  Chief  of  Staff  for  Research  and 
Development  at  Headquarters,  Department  of  the  Air  Force.  Loh  also 
served  as  the  commander  of  the  Aeronautical  Systems  Division,  the 
parent  organization  of  the  F-22  program  office,  as  the  program 
completed  demonstration/validation.  With  his  extensive  experience 
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Photo  courtesy  of  Lod<h©ed-Bo©liig  Team. 


F-22  prototype  is  44’-6’’  wide,  62’-r  long,  and  16’-5”  in  height.  Use  the  prototype  wi  lead  to  an  advanced  tactical  fighter 
balanced  design  of  low  observability,  agility,  supercruise,  and  advanced  offansi\«  and  defensive  aviontos. 
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and  multiple  points  of  view,  Loh  provides  an  important  and  valuable 

perspective  on  the  F-22  development  program. 

Customer  Satisfaction 

Query:  From  Air  Combat  Command’s  standpoint,  are  you,  the 

customer,  satisfied  with  what  you  are  seeing  on  the  F-22 
program? 

Loh:  Oh  yes.  I’m  satisfied  that  we  are  going  to  get  the  airplane 

that  we  signed  up  to... that  the  management  of  the  pro¬ 
gram  is  excellent... that  we,  the  customer,  have  a  direct 
input  into  the  program  as  part  of  the  integrated  product 
teams  [IPTs,]  and  that  our  voice  is  heard  all  of  the  time.  I 
have  no  problem  whatsoever  in  the  way  the  interaction 
between  customer  and  supplier  is  being  handled  today,  or 
ever,  in  the  F-22  program.  That  has  been  a  more  open 
process  than  any  other  program  I  can  recall.  It  continues 
to  be  a  very  open  process  and  I  attribute  that  to  the  fact 
that  we  do  have  this  IPT  process.  We  are  members  of  the 
IPT.  The  SPO  [System  Program  Office]  is  very  open  and 
candid  about  problems.... 

My  only  complaint  about  the  program  is  the  length  of 
the  development  program,  which  is  not  the  fault  of  the 
SPO  but... of  the  manner  in  which  we  [the  U.S.  Govern¬ 
ment]  allocate  resources  to  it  year  by  year.  It  is  extending 
the  development  program  year  by  year  beyond  a  most  effi¬ 
cient  level.  And  of  course  we  continually  have  to  look,  to 
the  future  and  see  what  is  out  there  for  us  to  face.  The 
flexibility  and  potential  of  the  F-22  program  to  spin  off 
other  derivative  aircraft  using  the  technology  we’ve  devel¬ 
oped  in  the  F-22  is  quite  high. 

'The  F-22  Program  as  a  Model 

Query:  Many  people  say  that  the  F-22  is  a  model  acquisition 

program.  Would  you  agree  with  that? 

Loh:  It  is  as  model  as  you  can  get.  I  attribute  that  to  the  fact 

that  we  have  had  the  IPT  process  from  the  very  begin¬ 
ning  and  that  includes  not  just  oiorselves  [Air  Combat 
Command]  and  the  SPO  but  also  the  contractor  team.  And 
I  attribute  it  to  the  fact  that  we  used  lessons  learned  from 
the  past....  When  we  put  the  program  together, we  didn’t 
encourage  excessive  risk.  I  attribute  where  we  are  today  as 
a  manifestation  of  aH  that,  in  that  we  have  a  program  that 
is  not  in  trouble  technically,  it  is  not  in  trouble  from  a 
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schedule  standpoint,  we  understand  its  costs,  and  there  is 
no  excessive  cost  growth  in  the  program.  It  is  a  well- 
managed  program  and  [is]  well  put  together.  So  if  that  is 
the  measure  of  a  model  program;  that  the  F-22  is  on  cost, 
on  schedule,  and  meeting  our  essential  performance  needs, 
it  is  indeed  a  model  program. 

The  View  From  Air  Force  Acquisition 

Air  Force  acquisition  views  on  F-22  development  are  expressed 
by  the  Principal  Deputy  Assistant  Secretary  for  Acquisition  and 
Management,  Darleen  A.  Druyun,  a  senior  leader  in  Air  Force 
acquisition  who  also  had  firsthand  experience  with  the  F-22  pro¬ 
gram.  During  the  1980s  and  early  1990s  Druyun  became  intimate¬ 
ly  involved  in  the  development  of  the  acquisition  strategy  for  the 
ATF  EMD  phase  in  the  Directorate  of  Contracting  at  Headquarters 
Air  Force  Systems  Command.  In  the  first  half  of  1991,  she  partici¬ 
pated  in  the  ATF  source  selection  for  EMD.  In  her  current  role  as 
the  Principal  Deputy  to  the  Air  Force  Acquisition  Executive  she 
observes  the  progress  of  the  program  monthly.  Her  long  history 
with  the  program  and  her  senior  position  within  Air  Force  acquisi¬ 
tion,  makes  Druyun  ideally  suited  to  assess  the  Air  Force  acquisi¬ 
tion  view  of  the  F-22  development  program. 

Customer  Satisfaction 

Query:  From  the  perspective  of  Air  Force  Acquisition,  has  the 

F-22  program  been  successful? 

Druyun:  The  program  itself  has  been  successful  and  continues  to  be 
successful  in  terms  of  how  we  are  implementing  it. 
However,  it  has  not  been  as  successful  as  I  had  hoped  with 
respect  to  the  budget  because  we  just  don’t  have  budget 
stability.  Every  single  year  it  seems  that  we  trim  $100 
million  here  or  $200  million  there  on  the  F-22  and  it 
becomes  a  bill-payer  for  other  requirements  in  the  Air 
Force.  We  tried  to  take  all  of  the  lessons  learned  in  struc¬ 
turing  the  EMD  contract  so  that  we  could  have  the 
contractor  come  up  with  his  IMP  [integrated  master  plan] 
and  IMS  [integrated  master  schedule]  and  have  him  stick 
to  that  schedule.  As  we  have  taken  the  money  out  of  the 
program  we  have  caused  that  program  to  stretch,  stretch, 
and  stretch.  It  is  there  where  we  have  not  been  successful. 

The  basic  structure  that  was  laid  in  place  to  manage 
that  program  is  working  exceptionally  well.  The  cost  and 
schedule  status  reports  show  we  are  within  about  3.5% 
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[of]  cost  and  about  2%  within  schedule.  This  is  with 
around  33%  of  the  program  complete.  If  we  were  going  to 
have  problems  on  this  program,  you  would  have  seen 
them  very  early,  and  they  haven’t  shown  up.  It  is  a  very 
successful  program  and  extremely  well  managed. 

The  F-22  Program  as  a  Model 

Query:  Many  have  called  the  F-22  acquisition  a  model  acquisition 

program.  Do  you  agree? 

Druyun:  I  felt  back  in  1990  and  1991  that  the  F-22  was  a  model 
program  because  General  Fain  came  up  with  the  idea  for 
an  integrated  master  plan  and  integrated  master  sched¬ 
ule.  You  take  all  of  the  documents  that  the  contractor  gives 
you  in  source  selection  and  use  them.  You  have  them  num¬ 
bered  the  same  way.  That  is  now  a  common  way  of  doing 
business  in  the  Air  Force.  You  see  that  in  any  major 
weapon  system  we  are  procuring.  Even  the  way  the  F-22 
contractor  submits  their  cost  and  pricing  data  is  in  the 
same  manner  as  how  they  write  their  statement  of  work. 
It  is  all  linked  together. 

The  F-22  became  the  pathfinder  for  how  to  do  acqui¬ 
sition  in  the  future.  All  of  the  programs  now  in  the 
Air  Force  are  using  the  ideas  developed  by  the  F-22. 
Today,  I  choose  to  call  the  F-22  program  a  pathfinder, 
as  opposed  to  a  model,  because  it  really  sets  the  key 
principles  firmly  in  place. 

The  Program  Director's  Perspective 

Major  General  Robert  F.  Raggio  succeeded  General  Fain  as  F-22 
Program  Director  on  July  1,  1992.  At  that  point,  the  EMD  phase  was 
not  yet  a  year  old.  The  implementation  of  the  integrated  manage¬ 
ment  framework  had  clearly  started  but  had  not  yet  matured. 
Raggio  explains  how  the  IPTs  and  integrated  tool  set  evolved  and 
how  well  they  worked. 

Integrated  Product  Teams 

Query:  How  have  IPTs  worked  out?  What  changes  have  you  made 

to  the  original  IPT  concept? 

Raggio:  First  get  it  straight — IPTs  are  nothing  more  than  a  tool 

for  integrated  product  development.  IPTs  gave  us  a 
straightforward  way  to  successfully  implement  integrat¬ 
ed  product  development. 

IPTs  have  been  incredibly  successful.  Each  team  has 
taken  a  balanced  approach  to  their  product  design 
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considering  all  of  the  elements,  such  as:  design,  manu¬ 
facturing,  near-term  cost  as  well  as  ownership  costs,  ease 
of  repair,  etc. 

IPX  leads,  both  on  the  contractor  side  and  the 
Government  side,  have  typically  come  from  an  engineer¬ 
ing  specialty.  As  a  result,  they  have  easily  focused  on  the 
technical  issues.  We  [both  Raggio  and  Gary  Riley,  the 
current  Lockheed  Program  Manager]  initially  had  to  force 
them  to  focus  on  the  cost  and  schedule  part  of  their 
programs.  We  have  worked  hard  to  make  the  IPX  leads 
into  mini“program  directors. 

For  IPTs  to  work,  they  need  buy-in  from  the  very  high¬ 
est  levels  on  both  the  Government  and  the  contractor  sides. 
Xhe  F-22  program  could  not  have  implemented  IPXs  unless 
Mr.  [Daniel]  Xelep,  Mr.  [Frank]  Shrontz  [Lockheed  and 
Boeing  chief  executive  officers],  General  [Ronald]  Yates 
[Commander  of  Air  Force  Materiel  Command],  and 
General  Loh  were  sold  on  IPXs.  A  functional  [specialty] 
could  have  slow-rolled  it — ^but  that  never  happened  since 
the  corporate  leadership  agreed  with  the  concept. 

We  found  that  we  had  to  work  carefully  with  the  func¬ 
tionals.  Some  felt  that  IPTs  would  reduce  their  power  base 
and  importance  to  the  organization.  In  fact,  we  have  found 
that  the  role  of  the  functionals  in  a  program  using  IPXs 
gets  greater,  not  less. 

A  problem  that  we  identified  early  on  was  that  some 
IPXs  became  independent  rather  than  integrated.  We 
found  that  once  they  were  empowered  to  design  their  prod¬ 
uct,  they  did  a  great  job,  except  they  designed  it  at  the 
expense  of  other  products.  In  the  old  system,  before  IPTfe, 
you  had  to  integrate  functional  specialties  efforts  in  design¬ 
ing  a  product.  Under  IPTs,  we  have  to  integrate  the  prod¬ 
uct  team  efforts  with  other  parts  of  the  weapon  system. 

We  originally  established  analysis  and  integration 
[A&I]  teams  to... make  sure  that  each  product  design 
meshed  with  the  rest  of  the  weapon  system.  We  initially 
set  them  in  the  organization  at  the  same  level  as  the 
IPTs.  This  was  not  sufficiently  effective.  We  adjusted  the 
integration  level;  now  our  A&I  teams  are  a  half-notch 
higher  than  the  IPTs,  so  that  they  can  direct  their  activ¬ 
ities  and  balance  the  teams. 

I  had  a  similar  problem  with  my  functionals  [e.g.. 
Engineering,  Contracting,  Finance]  and  my  product 
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teams  [e.g.,  Air  Vehicle,  Support  System].  I  set  up  a 
weapons  system  A&I  team  made  up  of  my  four  lead 
IPTs  and  all  of  my  functional  chiefs.  I  now  get  a  balanced 
perspective  on  all  issues. 

The  key  is  that  integration  must  be  balanced.  If  you 
force  integration  into  the  product  teams,  the  decisions  at 
the  lowest  levels  get  better,  and  developing  the  product 
becomes  much  easier. 

Integrated  Management  Tools 

Query:  How  has  the  integrated  tool  set  worked  out?  What 

changes  have  you  made  to  the  original  concepts? 

Raggio:  The  tool  set  has  been  a  high  payoff.  The  program  captures 

and  tracks  the  information  the  IPTs  need.  Generally, 
management  above  the  individual  IPT  reviews  just 
summary  data  from  the  IPTs.  Again,  the  key  is  that  these 
tools  are  integrated.  All  data  mutually  support  other  data. 

Work  breakdown  structure.  Establishing  the  work 
breakdown  structure  to  accurately  reflect  how  Lockheed 
and  Pratt  &  Whitney  are  designing  the  weapon  system,  and 
then  matching  the  organization  to  align  with  the  work 
breakdown  structure  along  IPT  lines,  has  allowed  full  inte¬ 
gration  across  the  program. 

Having  this  type  of  work  breakdown  structure  allows 
cost  performance  reports  to  flow  through  IPTs.  In  fact,  cost 
performance  reports  must  flow  through  the  IPTs.  The 
adaptation  of  the  standard  cost  performance  report  to 
include  a  report  based  on  IPTs  [the  Format  2B]  was  a 
great  idea.  This  report  shows  the  funding  by  product  and 
not  by  the  standard  functionaJ  shred. 

Early  on  we  thought  the  hardest  thing  to  do  was  get 
the  engineers  melded  with  the  program  managers.  That 
happened  pretty  quickly.  It  has  been  a  bigger  leap  to  get 
the  engineers  and  program  managers  to  meld  with  the 
financial  experts.  The  two  major  rephases  on  the  F-22 
(driven  by  Office  of  the  Secretary  of  Defense  or  congres- 
sionally  reduced  budgets)  have  better  forced  this  meld. 

Integrated  master  plan/integrated  master 
schedule.  If  the  cost  performance  report  were  perfect, 
there  would  not  be  a  need  for  an  IMP  or  an  IMS.  Since  the 
cost/schedule  control  system  has,  unfortunately,  evolved 
over  the  years  into  a  tool  typically  used  by  the  financial 
community,  and  not  so  much  by  program  managers,  non- 
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finance  personnel  don’t  use  it  and  don’t  put  the  effort  into 
the  plan  that’s  required.  We  used  IMP/IMS  to  force  the 
planning  and  networking  of  relationships.  It  was  critical 
to  have  this  when  we  set  up  our  critical  paths. 

Technical  performance  measures.  We  have  redis¬ 
covered  that  metrics  motivate  behavior.  The  careful  selec¬ 
tion  of  metrics  has  allowed  each  level  of  IPT  to  focus  on 
the  things  that  really  matter.  Many  TPMs  [technical 
performance  measures],  such  as  product  weight  or  design- 
to-cost,  roll  up  to  higher  levels  of  indentures,  like  an 
antenna  is  a  part  of  the  higher  level  avionics  system.  In 
this  way,  we  have  solid  visibility  into  overall  aircraft  para¬ 
meters,  like  weight.  These  types  of  metrics  give  us  an 
excellent  bottoms-up  prediction  of  performance. 

An  obvious  but  critical  lesson  we  learned  is  that  you 
need  to  ensure  that  you  are  gathering  quality  data  for  the 
technical  performance  measure  and  that  one  understands 
what  the  data  really  mean.  I  also  want  to  make  sure  that, 
if  at  all  possible,  every  metric  fits  the  standard  metric 
format  [shown  in  Figure  8-5].  If  it  does,  one  can  quickly 
determine  if  performance  is  good  or  bad. 

Legacy  of  the  F-22  Program 

Query:  What  do  you  feel  will  be  the  legacy  of  the  F-22  develop¬ 

ment  program? 

Raggio:  The  F-22  program  has  proven  beyond  a  shadow  of  a  doubt 

that  given  three  things:  (a)  requirements  stability,  (b) 
careful  up-front  planning  of  the  structure  of  the  program, 
and  (c)  funding  stability,  then,  you  can  deliver  on  your 
promises.  The  problem  is  that  we  have  enjoyed  the  first 
two,  but  not  funding  stability. 

We’ve  enjoyed  requirements  stability  because  General 
Loh  has  personally  worked  to  make  sure  that  the  require¬ 
ments  did  not  vary  significantly.  We  have  had  a  well- 
developed  plan  due  to  the  work  carried  out  in  demonstra¬ 
tion/validation.  Funding  stability  is  our  Achilles’  heel. 

The  main  problem  in  Government  acquisition  is 
systemic  of  our  type  of  government.  A  contractor  writes  a 
contract  with  the  executive  branch,  not  the  legislative 
branch.  The  legislative  branch  must  approve  funds  for 
that  contract  on  a  year-to-year  basis.  If  we  were  serious 
about  acquisition  reform,  we  would  figure  out  a  way  to  get 
the  legislative  branch  to  buy  into  the  same  plan  as  the 
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executive  branch  over  20  years,  or  at  least  5  to  6  years. 
The  technical  problems  that  have  occiirred  on  F-22 
are  well  in  control.  They  have  been  handled  well  by  our 
system,  the  IPTs  with  the  integrated  tools,  and  our  people. 
You  can  see  the  payoff  of  a  well-structured,  carefully 
planned,  carefully  executed  program  in  how  we  are  doing 
in  our  cost  and  schedule  performance  as  tracked  by  the 
C/SCS  [cost/schedule  control  system],  Mr.  Gary  Christie 
[the  Office  of  the  Secretary  of  Defense  official  responsible 
for  tracking  the  cost  and  schedule  performance  of  all 
Department  of  Defense  contracts]  told  me  that  the  F-22 
program  is  doing  exactly  what  he  has  always  wanted — lay 
in  cost  performance  report  data  in  detail  down  to  the 
lowest  level  of  the  program.  Gary  says  he’s  not  seen  anoth¬ 
er  program  like  this.  We  are  an  $11.2  billion  cost  contract 
that  is  around  33%  complete,  and  we  are  under  2%  sched¬ 
ule  variance  and  under  3.5%  cost  variance.  And  what  is 
amazing,  if  we  had  had  stable  funding,  we  could  have  done 
even  better!  [See  Figure  9-1  for  a  history  of  Lockheed’s 
cost  and  schedule  variance  on  the  weapon  system  contract 
and  for  Christie’s  views  on  the  F-22  program.] 

A  drawback  of  a  very  tightly  planned  program,  as  tightly  planned 
as  the  F-22  program  is,  is  that  the  program  cannot  adjust  very  easily 
to  changes  in  funding  levels.  A  conclusion  on  the  F-22  program  may  be 
that  if  the  system  wants  a  well-structmed  program,  it  may  not  be  able 
to  take  the  current  sloppy  funding  process. 

Visible  Advantages  of  the  F-22  Program 

Raggio  believes  that  IPD,  IPTs,  and  the  tools  and  procedures  of  the 
integrated  management  framework  are  powerfiil  components  of  a  well- 
run  program.  He  cited  three  tangible  results  of  the  F-22  program 
method  of  operation:  improved  design  maturity,  increased  user  involve¬ 
ment  in  design  solutions,  and  improved  management  flexibility. 

Improved  Design  Maturity 

The  F-22  program  framework  of  IPTs  using  integrated  tools 
results  in  a  more  balanced  design  because  the  perspectives  of  not  only 
design  engineers,  but  also  systems  engineers  (who  look  at  the  entire 
weapon  system),  manufacturing  engineers,  and  maintenance  and 
logistics  personnel  (who  assess  the  ease  of  repairing  and  supporting 
the  weapon  system)  influence  the  initial  design.  For  example,  in 
December  1993,  Lockheed  developed  and  introduced  a  new  tool  and 
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method  for  computing  the  aircraft  radar  signature.  The  initial  results 
showed  that  minor  changes  in  the  aircraft  design  from  the  full-scale 
signature  pole  model  used  in  demonstration/validation  resulted  in 
greater  increases  in  signature  than  previously  assessed.  If  this  prob¬ 
lem  was  not  addressed,  the  aircraft  would  not  fulfill  some  signature 
requirements.  The  F-22  development  team  responded  by  asking  all 
sub-IPTs  whose  products  contributed  to  the  aircraft  overall  radar 
signature  to  revisit  their  product  design  to  see  if  they  could  lower  their 
contribution  to  the  aircraft  radar  signature.  In  trying  to  solve  the  prob¬ 
lem,  each  team  assessed  the  effect  of  their  solution  not  just  on  signa¬ 
ture  reduction  but  also  on  the  ability  to  manufacture  the  product,  the 
ease  of  maintaining  the  product  or  overall  aircraft,  and  the  overall  life- 
cycle  cost  of  the  product.  Because  of  the  LPT  process,  a  potentially 
major  problem  was  solved  by  means  of  a  balanced  design  that  met  the 
challenges  of  radar  signature  reduction  while  maintaining  the  other 
weapon  system  requirements  (for  example,  ease  of  maintenance,  pro- 
ducibility).  Creating  a  balanced  design  early  reduces  the  need  to 
change  the  design  later  in  the  program.  In  earlier  programs,  manu¬ 
facturing  engineers  typically  had  to  adjust  the  design  to  build  the 
product;  they  don't  need  to  in  the  F-22  program  because  they  helped  in 
the  initial  design.  As  a  resiilt,  the  balanced  design  of  the  F-22  is  also  a 
mature  design  that  will  require  fewer  changes  at  production. 

User  Involvement  in  Design  Solutions 

The  power  of  teaming  is  that  all  the  participants  understand  each 
other’s  constraints.  The  F-22  EMD  phase  (like  the  demonstration/ 
validation  phase)  has  exposed  the  operational  user  to  the  real-world 
limitations  of  design  and  production.  Every  major  design  trade-off*  and 
refinement  (and  most  minor  ones)  have  involved  the  members  of  Air 
Combat  Command.  As  a  result,  the  participants  understand  the 
issues  and  thus  imderstand  why  the  team  had  to  change  some  aspect 
of  the  design.  As  Loh  commented,  user  involvement  resvdts  in  very 
high  user  satisfaction.  Air  Combat  Command  is  pleased  with  the  prod¬ 
uct  now  under  development. 

Improved  Management  Flexibility 

A  major  reason  for  establishing  IPTs  was  to  identify  problems 
earlier  than  in  previous  programs.  Three  of  the  benefits  Raggio  has 
identified  are 

•  Earlier  identification  of  problems 

•  Faster  agreement  on  solutions 

•  Better  response  time  in  contingencies. 
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Figure  9-1.  Cost  and  Schedule  Variance  for  the 
Lockheed  F-22  Weapon  System  Contract 


Cost  Variation  %  Schedule  Variation  % 


Source;  F-22  Program  Office 


Mr.  Gary  Christie  on  Cost/Schedule  Control  System 

We  have  an  acquisition  initiative  to  reform  the  way  the  Department  of  Defense’s 
programs  implement  the  earned  value  portion  of  the  cost/schedule  control  sys¬ 
tem.  We  stress  three  things; 

1.  Earned  value  is  a  program  management  tool,  not  just  a  financial  tool, 

2.  The  program  must  be  planned  out  In  thorough  detail  down  to  the  cost 
account  level,  and 

3.  The  program  must  have  integrated  schedules,  and  other  management 
tools,  which  are  integrated  vertically  (up  and  down  the  organization)  and  horizon¬ 
tally  (with  other  product  teams  or  other  specialties....) 

...I  think  with  their  IMP/IMS  and  other  Integrated  tools,  the  F-22  Is  doing 
what  we  think  they  ought  to  be  doing  to  run  a  successful  program.  I  am  not 

ready  to  say  you  should  ‘cookie  cutter’  the  F  22  approach  for  every  program,  but 
the  two  keys  are  there;  It’s  got  a  detailed  plan  and  it’s  integrated. 

The  F-22  Is  executing  unusually  well....  We  generally  consider  the  two  best 
executing  development  programs  in  the  Department  today  are  the  F-22  and  the 
F/A-18E/F.  If  you  look  at  the  F/A-18E/F  program,  you  will  see  they  have  imple¬ 
mented  many  concepts  from  the  F-22,  such  as  integrated  product  teams,  inte¬ 
grated  schedules,  and  a  tight  tracing  system  for  earned  value.” 

Gatv  ChristJe,  Office  of  the  Secretary  of  Defense, 
interview  by  author,  tape  recording,  Washington.  DC.  May  16.  1995 
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Collocating  multifunctional  experts  in  product-focused  teams  and 
empowering  them  to  manage  all  aspects  of  their  product  results  in 
strong  product  ownership.  The  teams  are  motivated  to  find  problems 
that  are  hindering  the  successful  design  and  production  of  their  prod¬ 
uct  and  work  hard  to  find  the  best  solution  that  falls  within  everyone’s 
constraints.  Higher  level  IPTs  and,  ultimately,  the  Progreun  Director 
must  ensure  the  IPTs  understand  all  of  the  higher  level  constraints 
they  must  consider  (the  purpose  of  the  analysis  and  integration  teams). 

By  identifying  problems  earlier,  getting  rapid  agreement  for  the 
right  course  of  action,  and  responding  rapidly  to  changing  conditions, 
IPTs  have  given  the  F-22  program  management  much  greater  flexi¬ 
bility  to  respond  to  problems  and  other  unforeseen  challenges. 

Lessons  Learned 

The  history  of  acquisition  is  characterized  by  dedicated  individ¬ 
uals  who  learn  from  earlier  programs.  After  4  years  of  experience 
with  the  new  tools  and  methods,  what  can  be  learned  from  the  F-22 
program?  The  following  observations  come  from  the  program  office, 
various  studies,  and  my  own  work  in  the  program  office  and  research. 
Figure  9-2  summarizes  the  lessons  learned. 

The  Program 

1.  Full  teamwork,  among  the  operator,  the  program  office, 
and  the  contractor,  improves  overall  program  performance.  In 

the  demonstration/validation  phase  the  requirements  refinement 
process  provided  the  best  example  of  teamwork.  Today,  as  demon¬ 
strated  by  Captain  Tinsler’s  memo  reproduced  in  Figure  7-7,  the 
day-to-day  actions  of  the  IPTs  epitomize  teamwork. 

2.  Integrated,  detailed  planning  before  the  start  of  the 
program  helps  reduce  start-up  time  and  supports  a  well- 
managed  program.  For  example,  the  program  office  required  the 
competing  contractors  to  submit  their  proposed  IMP  and  IMS  as  part 
of  their  proposals  for  EMD  (see  appendix  D),  and  the  contractors 
invested  great  amounts  of  time  and  thought  into  developing  the  IMP 
and  IMS  because  they  were  part  of  the  evaluation  process.  Unlike 
typical  programs,  which  take  the  first  6  to  12  months  to  plan  and  get 
organized,  the  F-22  and  F119  teams  went  immediately  to  work. 

3.  Integrated  management  tools  support  a  disciplined  man¬ 
agement  process.  The  integrated  management  framework  provides 
coherent  information  for  all  team  members,  including  the  program 
leadership.  The  IMP  and  IMS  provide  a  detailed  yet  flexible  map  for 
the  program  to  follow.  These  tools,  coupled  with  strong  leadership, 
result  in  a  disciplined  development  program. 
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4.  Tightly  managed  programs  can  deliver  on  their  promis¬ 
es  but  suffer  greatly  from  unstable  funding.  As  Raggio  pointed 
out,  funding  instability  has  been  the  F-22  Achilles'  heel.  Reductions  in 
such  a  well-scheduled,  well-structured  program  leave  the  program 
director  two  options:  reduce  performance  or  delay  the  delivery  of  the 
weapon  system.  The  Air  Force  has  chosen  the  second  option.  How¬ 
ever,  slipping  the  program  creates  a  great  deal  of  turbulence.  In  1994 
and  1995,  as  a  result  of  budget  reductions,  many  of  the  program  office 
personnel  and  senior  contractors  spent  over  half  of  their  time  resched¬ 
uling  program  work  to  match  the  budget — time  that  the  team  could 
not  spend  improving  the  quality  of  the  product. 

Integrated  Product  Development 

5.  Successful  implementation  of  IPD  requires  complete 
senior-level  support  and  encouragement  from  all  key  players. 

IPD  is  sufficiently  different  from  the  classical,  functional  way  of  oper- 


Figure  9-2.  Lessons  Learned  From  the  F-22  Program 


Program 

1.  Full  teamwork  improves  program  performance. 

2.  Integrated  planning  reduces  start-up  and  supports  a  well- 
managed  program. 

3.  Integrated  management  tools  support  a  disciplined  process. 

4.  Tightly  managed  programs  suffer  greatly  from  unstable  funding. 

Integrated  Product  Development 

5.  IPD  requires  complete  senior-level  support. 

6.  IPD  requires  a  cultural  change. 

7.  Classic  functional  organizations  may  initially  resist  IPD. 

Integrated  Product  Teams 

8.  IPTs  naturally  accomplish  IPD. 

9.  IPT  members  are  motivated,  are  involved,  and  have  ownership. 

10.  Every  IPT  requires  training. 

11.  IPTs  take  leadership  commitment  from  all  groups. 

12.  IPTs  should  have  experienced  and  empowered  members 

13.  The  I  in  IPT  can  easily  become  independent  instead  of  integrated. 

14.  IPTs  need  to  know  their  fenceposts. 

15.  Program  leadership  must  ensure  integration  across  the  IPTs  at 
every  level. 

16.  Communication  and  software  tools  help  integration. 

17.  IPT  managers  should  be  in  charge  of  personnel  and  budget 
resources. 

18.  Both  leadership  and  teams  must  set  and  track  IPT  objectives. 

19.  The  right  people  must  be  in  the  right  Jobs  at  the  right  time. 
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ating  that  senior  management  must  commit  to  and  direct  the  change. 
Moving  individuals  from  their  traditional  functional  organizations 
toward  integrated  teams  requires  the  sanction  of  top  management. 

6.  Successful  implementation  of  IPD  requires  a  cultural 
change^  which  does  not  occur  overnight.  To  make  the  transi¬ 
tion  from  thinking  about  an  item  from  a  functional  perspective  to 
thinking  about  it  from  an  integrated  product  perspective  requires 
that  individual  workers  look  at  the  development  process  in  a  new 
way.  Part  of  this  change  comes  with  focusing  on  the  product  and 
part  from  focusing  on  life-cycle  requirements.  To  fully  implement 
IPD  requires  a  fundamental  change  in  individual  and  organization¬ 
al  perspectives. 

Such  a  cultural  change  is  not  without  its  costs,  as  not  all  individ¬ 
uals  are  comfortable  with  the  IPD  philosophy.  The  F-22  program  lead¬ 
ership  needed  to  move  several  mid-level  managers,  both  Government 
and  contractor  employees,  to  other  programs  because  they  did  not 
adapt  to  IPD. 

7.  Classic  functional  organizations  may  initially  resist  IPD. 

This  problem  requires  top-level  attention.  In  some  locations,  function¬ 
al  leaders  tended  to  resist  the  implementation  of  IPD.  Firm  backing 
and  guidance  from  the  highest  Government  and  corporate  leadership 
eliminated  this  issue. 

Integrated  Product  Teams 

8.  IPTs,  when  given  the  focus  of  working  life-cycle  issues, 
naturally  accomplish  IPD.  IPTs  exist  solely  to  support  IPD. 
Collocating  the  necessary  individuals  with  the  required  skills  and 
perspectives  and  giving  them  the  task  of  developing  a  long-term 
solution  leads  to  IPD. 

9.  When  the  program  leadership  assigns  individuals  to 
teams  and  holds  them  responsible  for  the  successful  develop¬ 
ment  of  the  product,  the  result  is  strong  worker  motivation, 
involvement,  and  ownership.  A  survey  conducted  in  1993  of  more 
than  70  percent  of  Government  F-22  program  office  personnel  found 
that  the  great  majority  of  the  members  of  IPTs  believed  that  they  per¬ 
sonally  made  a  difference  in  the  development  of  their  product.  Over  80 
percent  believed  that  the  quality  of  the  F-22  weapon  system  would  be 
improved  through  the  use  of  the  product  team  approach.  One  respon¬ 
dent  concluded,  “There  is  inherently  more  buy-in  by  those  responsible 
for  the  product  due,  in  part,  to  the  higher  degree  of  accountability.” 

10.  Every  level  of  DPT  requires  training.  Initially  the  F-22 
program  indoctrinated  each  team  member,  Grovemment  and  contrac¬ 
tor,  with  the  concepts  of  concurrent  leadership  and  IPD  to  help 
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guarantee  that  they  all  knew  what  was  expected  of  them.  In  addition, 
the  program  senior  leadership  from  the  primary  contractors  and  the 
Government  initially  met  for  4  days  to  discuss  commimications,  expec¬ 
tations,  and  methods  of  operation,  which  helped  the  different  segments 
of  the  team  understand  the  concerns  of  the  other  team  members  and 
work  together.  Individual  IPTs  met  as  well  to  discuss  a  multitude  of 
issues  including  the  teaming  environment,  intragroup  dynamics,  and 
the  way  the  contractor/Govemment  teams  could  best  use  the  integrat¬ 
ed  tools.  This  training,  which  the  program  regularly  repeats  for  new 
members,  has  helped  greatly  to  establish  the  needed  cultural  change  to 
support  IPD  and  high-performance  integrated  teams. 

11.  Establishing  IPTs  takes  commitment  from  the  top  on 
the  part  of  the  Grovemment  and  contractor.  As  mentioned  above, 
management  must  support  IPD.  To  organize  IPTs  requires  manage¬ 
ment  to  support  taking  experts  out  of  their  traditional  functional 
offices.  One  can  find  literally  hundreds  of  barriers  to  stop  IPTs.  Senior 
management  commitment  is  the  fastest,  most  effective  way  to  elimi¬ 
nate  these  barriers. 

12.  IPTs  should  have  experienced  and  empowered  mem¬ 
bers.  This  is  the  potential  shortcoming  of  IPTs.  In  traditional  organi¬ 
zations,  experts  sit  with  other  experts  in  the  same  function.  New  mem¬ 
bers  learn  fi-om  these  experts  and,  when  confronted  with  something 
new,  can  easily  ask  someone  from  the  office.  In  IFIk,  because  the  fimc- 
tional  member  of  the  team  will  frequently  be  the  only  person  from  that 
specialty,  that  person  needs  the  experience  and  knowledge  to  resolve 
the  problem  without  calling  back  to  the  functional  office.  For  specialists 
to  be  truly  effective,  the  team,  as  well  as  the  functional  office,  must 
empower  the  IPT  members  to  use  their  judgment  on  their  product. 

13.  The  I  in  IPT  can  easily  become  independent  instead  of 
integrated*  The  first  lesson  of  the  F-22  EMD  phase  was  that  empow¬ 
ered  teams  with  allocated  requirements,  budget,  and  schedule  tended 
to  create  their  own  optimized  product — at  the  expense  of  the  rest  of  the 
weapon  system.  The  program  leadership  responded  to  this  problem  by 
strengthening  the  role  of  the  analysis  and  integration  team,  whose 
missions  changed  seemingly  overnight  from  primarily  one  of  analysis 
to  product  integration.  As  Raggio  mentioned,  he  further  strengthened 
these  analysis  and  integration  teams  by  raising  their  stature  to  be  a 
half-notch  higher  than  the  product  teams  they  integrate. 

14.  EPTs  need  to  know  their  fenceposts.  IPTs  need  to  under¬ 
stand  what  decisions  they  can  make  and  what  decisions  they  must 
pass  to  a  higher  level  IPT  or  even  the  program  director.  If  the  IPT 
higher  level  management  properly  sets  the  team  decision  space,  the 
chance  of  “independent”  IPTs  is  reduced. 
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15.  The  program  leadership  must  ensure  integration 
across  the  IPTs  at  every  level.  Integration  across  IPTs  prevents 
‘‘independent^  product  development  and  helps  ensure  efficient, 
system-level  weapon  development. 

16.  A  network  of  communication  and  software  tools  that 
ties  together  all  team  members  improves  integration.  The 
F-22  program  implemented  a  teamwide  management/technical 
information  system  that  allows  both  Government  and  contractor 
team  members  to  access  the  same  data  bases  (to  look  at,  for  exam¬ 
ple,  TPMs,  cost  status,  and  document  revisions)  and  to  communicate 
through  electronic  mail.  Standardized  software  products  allow  team 
members  to  transfer  files  between  locations,  and  an  encrypted  video 
teleconference  facility  allows  teams  at  the  various  contractor  and 
Government  sites  to  conduct  a  meeting  from  up  to  four  different 
locations. 

17.  IPT  managers  should  have  authority  over  personnel 
and  budget  resources.  As  in  any  organization,  a  leader  needs  to  have 
control  over  the  people  and  resources  that  make  up  the  team. 

18.  The  program  leadership  and  the  teams  themselves 
must  jointly  establish  and  track  IPT  goals  and  objectives. 
Typically  the  teams  set  goals  and  objectives  based  on  the  allocated 
requirements,  budget,  and  program  schedule.  The  IPT,  and  higher 
level  management,  tracks  its  performance  through  the  use  of  TPMs, 
To  ease  this  process,  the  leadership  must 

•  Ensure  that  the  appropriate  team  members  participate  in  the 
appropriate  decisions  so  that  their  expertise  flows  into  the  deci¬ 
sion  and  the  team  member  comes  away  fi-om  the  decision  with  a 
stronger  feeling  of  ownership  of  the  outcome. 

•  Develop  meaningful  metrics.  A  famous  program  office  saying 
goes:  “What  gets  tracked,  gets  done.”  TPMs  are  a  powerful  tool, 
and  the  IPT  leadership  must  make  sure  the  metrics  support  their 
key  activities  and  key  requirements. 

19.  The  program  leadership  should  put  the  right  people  in 
the  right  jobs  at  the  right  time.  Everyone  has  strengths  and  weak¬ 
nesses.  The  challenge  of  the  leader  is  to  place  individuals  in  positions 
that  make  full  use  of  their  strengths  and  minimize  exx>osure  of  their 
weaknesses.  To  do  so,  leaders  must: 

•  Appoint  and  train  leaders.  As  is  true  of  any  organization,  training 
for  IPT  leaders  must  stress  the  skills  needed  for  teamwork  and 
problem  solving. 

•  Replace  leaders  who  don't  or  can't  lead.  Not  all  individuals  are 
comfortable  with  IPTs.  The  best  technical  product  expert  may  not 
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be  the  best  leader.  The  interesting  twist  to  IPTs  is  that  the  best 
IPT  leader  may  be  a  contracting  expert,  a  finance  specialist,  or  a 
logistics  expert.  Traditionally,  individuals  from  these  and  other 
fields  would  never  get  a  chance  to  lead  a  product  development. 
Under  IPTs,  they  can. 


Summary 

Has  the  F>22  program  been  successful?  According  to  major  cus¬ 
tomers  of  the  program,  it  certainly  has  been  successful.  Has  the  use  of 
the  F-22  integrated  management  framework  and  IPTs  been  a 
panacea?  Certainly  not.  However,  these  tools  and  the  integrated 
development  approach  support  a  disciplined,  well-managed  program. 


Photo  courtosy  of  Lockheed-Martin. 


The  forward,  mid,  and  aft  fuselages,  with  wings,  have  been  joined  in  the  mate 
tool  In  construction  of  the  first  F‘22. 


Part 


Beyond  the  F-22 

One  of  the  ongoing  challenges  for  the  F-22  program  managers 
(both  at  the  program  office  and  at  contractor  sites)  is  to 
explain  the  process  they  use  to  run  the  program.  The  F-22 
Program  Director  and  his  industry  counterpart  speak  on  this  issue 
annually  to  the  Program  Management  class  at  the  Defense  Systems 
Management  College.  The  F-22  program  also  supports  Pentagon 
conferences  on  how  to  improve  the  process  of  acquiring  military 
hardware.  Since  1991,  the  F-22  program  leadership  has  presented 
explanations  of  integrated  product  development  through  the  use  of 
integrated  product  teams,  user/acquirer-refined  requirements, 
event-based  schedules,  measurement  of  technical  progress  through 
metrics,  and  the  other  topics  discussed  in  these  pages.  Thus,  these 
concepts  have  spread  and  been  refined  and  adapted  to  other  pro¬ 
grams  and  purposes.  Chapter  10  shows  how  the  concepts  have  been 
applied  outside  the  F-22  program.  Chapter  11  reviews  the  major 
points  of  the  previous  chapters. 
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I  am  directing  a  fundamental  change  in  the  way  the  Department 
acquires  goods  and  services.  The  concepts  ofIPPD  [integrated  process 
and  product  development]  and  IPTs  [integrated  product  teams]  shall 
be  applied  throughout  the  acquisition  process  to  the  maximum  extent 
practicable. 


William  J.  Perry, 
Secretary  of  Defense 


The  F-22  principles  and  concepts  (the  basic  acquisition  truths) 
apply  to  other  acquisition  efforts.  Darleen  A.  Dru3ain  of  the 
Office  of  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition 
has  commented  that  Air  Force  weapons  development  programs  are 
using  the  ideas  developed  by  the  F-22.  Integrated  product  develop¬ 
ment  (IPD)  (including  the  use  of  integrated  product  teams,  IPTs), 
product  performance  metrics  (such  as  technical  performance  mea¬ 
sures,  TPMs),  and  the  use  of  an  event-based  schedxile  with  success 
criteria  are  common  on  new  Air  Force  programs  such  as  the  Air  Force 
Joint  Direct  Attack  Munition,  the  Spaced-Based  Infrared  System, 
and  the  Enhanced  Expendable  Launch  Vehicle.  But  do  the  F-22  basic 
acquisition  truths  apply  elsewhere?  Where  have  the  F-22  program 
principles  and  concepts  migrated?  The  F-22  program  ideas  have 
spread  to  more  than  just  new  acquisition  programs:  they  are  in  use 
in  a  technology  program  (JAST),  an  existing  program  (C-17),  all  Navy 
programs,  and  the  Air  Force  Requirements  Oversight  Council.  The 
Department  of  Defense  (DOD)  plans  to  use  several  of  these  ideas  to 
help  improve  its  acquisition  process. 

The  Joint  Advanced  Strike  Technology  Program 

The  Joint  Advanced  Strike  Technology  (JAST)  program,  now  the 
Joint  Strike  Fighter  Program,  is  a  joint  Air  Force/Navy/Marine  Corps 
research  and  development  effort  that  is  exploring  affordable  techno¬ 
logy  and  approaches  for  next-generation  tactical  attack  aircraft. 
Lieutenant  General  George  K  Muellner,  the  Director  of  JAST,  worked 
closely  with  the  F-22  program  when  he  served  as  the  Deputy  Director 
of  Requirements  at  Tactical  Air  Command.  As  he  established  the 
JAST  program,  Muellner  looked  at  the  F-22  program  for  lessons 
learned.  In  his  view,  the  key  points  of  the  F-22  program  included  its 
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disciplined  process,  its  focus  on  the  effectiveness  of  the  weapon  system 
(as  opposed  to  a  focus  on  meeting  a  certain  parameter,  like  speed),  and 
its  use  of  IPTs  to  implement  IPD.  He  noted  that  all  JAST  contractors 
use  integrated  teams.  According  to  Muellner,  the  use  of  IPTs  is  now 
the  common  way  of  doing  business  throughout  the  aerospace  industry. 
In  addition,  two  specific  elements  of  the  F-22  program  that  Muellner 
brought  to  the  JAST  effort  were  modeling  and  performance  tracking. 

Muellner  saw  firsthand  that  the  ATF  program  modehng  approach 
allowed  a  fair  comparison  between  competing  solutions  (i.e.,  the 
Lockheed  F-22  and  the  Northrop  F-23)  that  could  successfully  stay 
focused  on  top-level  performance.  As  in  the  ATF  program,  he  struc¬ 
tured  the  JAST  program  to  require  all  competing  teams  to  use  a  stan¬ 
dard  version  of  the  tactical  performance  model,  TAG  BRAWLER 
(described  in  chapter  6).  In  fact,  the  JAST  program  has  expanded  on 
the  concept  of  benchmarked,  standardized  modeling  and  simulation  to 
forecast  and  compare  values  other  than  tactical  performance. 

The  JAST  program  has  also  used  the  F-22  concept  of  performance 
monitoring.  It  has  tracked  progress  on  established  key  variables  of 
performance  using  metrics  similar  to  TPMs.  These  metrics  allow  the 
JAST  program  office  to  monitor  the  technical  performance  of  the  var¬ 
ious  competing  portions  of  the  program  and  assess  technical  progress. 

The  JAST  program  studied  the  F-22  program  (as  well  as  less  suc¬ 
cessful  programs  like  the  Navy  A- 12)  and  tailored  many  of  its  concepts 
and  their  use  to  the  specific  JAST  application.  The  JAST  example 
shows  that,  as  well  as  working  with  new  programs,  the  F-22  program 
ideas  also  work  for  technology  development  efforts. 

The  C-17  Program  Recovery 

Could  an  established  program,  already  well  into  development, 
put  the  F-22  program  ideas  into  practice?  In  February  1993,  the 
Assistant  Secretary  of  Defense  for  Acquisition  and  Technology,  John 
M.  Deutch,  wanted  to  assess  whether  the  troubled  C-17  airlifter 
program  could  be  saved.  The  C-17  was  to  improve  the  Air  Force  abil¬ 
ity  to  move  cargo,  especially  outsized  cargo  that  can  usually  only 
fit  in  the  belly  of  a  C-5,  and  help  reduce  a  serious  shortfall  in  U.S. 
airhft.  One  of  the  many  improvements  the  C-17  was  to  bring  to  the 
Air  Force  airlifters  was  its  ability  to  land  on  small,  remote  airfields. 
The  program,  started  in  the  early  1980s,  had  run  into  many  prob¬ 
lems.  In  fact,  the  situation  became  so  tense  that  the  contractor, 
McDonnell-Douglas  Aircraft,  and  the  Government  were  preparing  a 
number  of  major  claims  to  file  against  one  another.  Faced  with  a 
critical  military  requirement  for  airlift,  Deutch  wanted  to  see  if,  and 
how,  DOD  could  recover  the  C-17  program. 
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Deutch  established  a  Defense  Science  Board  Task  Force  to  see 
what  the  problems  in  the  C-17  program  were  and  whether  they  coiild 
be  solved.  On  the  Task  Force  he  wanted  to  have  a  mix  of  experts  in  the 
world  of  acquisition  from  industry,  civilian  government,  and  the  mili¬ 
tary.  Appointed  as  co-chairs  for  this  study  were  Robert  A.  Fuhrman, 
previously  president  of  Lockheed  Aerospace  and  program  manager  for 
the  original  C-5  program,  and  Lieutenant  General  Fain,  Commander 
of  the  Aeronautical  Systems  Center,  and  a  former  F-22  program  direc¬ 
tor.  Also  serving  were  Edward  C.  "Pete”  Aldridge,  chairman  of  the 
Aerospace  Corporation  and  former  Secretary  of  the  Air  Force;  Oliver 
C.  Boileau  of  the  Northrop  Corporation;  and  Malcolm  R.  Currie, 
previously  of  Hughes  Aircraft  Company.  Nora  Slatkin,  who  was  then 
serving  as  Deutch's  special  assistant,  was  to  work  with  this  group  and 
follow  its  progress. 

Looking  back  to  his  experience  on  the  F-22  program,  Fain 
suggested  to  Fuhrman  that  they  establish  multifunctional  teams  to 
study  seven  critical  aspects  of  the  C-17  program: 

1.  Systems  engineering  and  operational  requirements 

2.  Supportability 

3.  Production  transition  and  manufacturing  processes 

4.  Ground  and  flight  testing 

5.  Financial  management 

6.  Contracting 

7 .  Program  management. 

Each  team  included  members  from  the  Air  Force,  the  appropri¬ 
ate  Office  of  the  Secretary  of  Defense  (OSD)  functional  area,  the 
DOD  Inspector  General  staff,  and  the  Defense  Contract  Manage¬ 
ment  Command.  These  teams  were,  in  fact,  IPTs. 

After  a  detailed  2-month  review,  the  Task  Force  found  that  the 
C-17  program  was  critical  for  the  Nation  and  that  the  program  could 
and  should  be  saved.  Among  many  recommendations,  the  Program 
Management  team  recommended  several  features  that  resembled 
the  F-22  program.  These  recommendations  included: 

•  Introduce  IPD  with  a  new  organization  based  on  multidiscipli¬ 
nary  teams  focused  on  products 

•  Create  an  integrated  tool  set  that  included  an  event-driven 

master  program  plan,  a  tiered,  integrated  master  schedule,  and 

standardized  metrics  with  TPMs. 

These  and  other  measures  became  part  of  the  agreement  between 
McDonnell  Douglas  Aircraft  and  DOD  to  save  the  C-17  program. 

In  1996  the  C-17  reached  initial  operational  capability,  and  Air 
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Mobility  Command  uses  the  new  aircraft  regularly  to  fly  airlift 
missions.  McDonnell  Douglas  has  increased  the  delivery  rate  of  C-17s 
and  is  actually  delivering  them  early.  At  a  follow-up  meeting  of  the 
Task  Force  to  assess  the  progress  of  the  C-17  recovery,  the 
Commander  of  the  Defense  Plant  Representative  Office  at  Douglas 
Aircraft,  Colonel  James  Klutter,  expressed  his  amazement  that  the 
C-17  Government/  Industry  team  had  turned  into  a  high-performance 
team.  There  are  many  reasons  for  this  great  success,  but  clearly  the 
use  of  empowered,  multifunctional  IPTs  using  integrated  tools,  like  an 
event-driven  master  plan  with  a  tiered,  integrated  master  schedule, 
has  been  a  great  help.  Note  that  the  basic  acquisition  truths  applied 
to  the  C-17  program,  even  though  the  program  had  been  underway  for 
over  10  years,  and  that  the  principles  of  IPTs,  also  applied  to  the 
Defense  Science  Board  review  process  itself. 

Navy  Interest 

In  February  1994,  a  year  following  the  start  of  the  C-17  Defense 
Science  Board  review,  the  Assistant  Secretary  of  the  Navy  for 
Research,  Development,  and  Acquisition,  Nora  Slatkin,  formerly 
Deutch’s  special  assistant,  arranged  for  Fain  to  explain  his  view  of 
the  acquisition  process,  primarily  the  concepts  and  ideas  developed 
when  he  led  the  ATF/F-22  program,  to  all  of  the  Navy  program  exec¬ 
utive  officers  and  program  managers.  On  February  15,  1994,  Fain 
explained  the  concepts  he  had  seen  work  so  well:  a  requirements 
evolution,  as  led  by  the  operational  user;  the  importance  of  matur¬ 
ing  technology  to  match  the  requirements;  the  value  of  IPD  using 
IPTs;  and  the  use  of  an  integrated  tool  set,  including  metrics. 

In  Navy  programs  such  as  the  New  Attack  Submarine  and  the 
new  amphibious  assault  ship,  the  Landing  Platform  Dock  17,  the  use 
of  IPTs  is  standard.  Many  of  the  other  concepts  Fain  presented  are 
also  being  institutionalized  throughout  Navy  acquisition,  showing 
that  the  basic  acquisition  truths  of  the  F-22  program  apply  to  other 
services  as  well. 

The  Air  Force  Requirements  Oversight  Council 

The  Air  Force  has  used  the  IPT  concept  to  improve  its  require¬ 
ments  process.  In  mid-1996.  Major  General  David  McCloud,  the 
Director  of  Operational  Requirements  for  the  Air  Force,  was  respon¬ 
sible  for  coordinating  and  reviewing  all  requirements  for  Air  Force 
weapon  systems.  A  recent  major  change  in  the  Air  Force  require¬ 
ments  process  came  directly  from  his  experience  on  the  ATF  program. 

Late  in  the  demonstration/validation  phase,  then  Colonel  McCloud 
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led  the  Tactical  Air  Command  division  responsible  for  establishing 
ATF  requirements.  The  essential  task  was  to  successfully  challenge 
operational  requirements  and  understand  their  cost.  Individuals  from 
the  program  office  and  from  McCloud's  office  would  cloister  themselves 
in  a  conference  room,  debate  the  merits  of  the  requirements,  and 
discuss  the  various  designs  necessary  to  reach  those  requirements.  As 
a  result  of  this  healthy  challenge-and-debate  cycle,  the  Air  Force  devel¬ 
oped  very  solid  requirements  for  the  ATF  that  balanced  operational 
need  against  platform  complexity  and  cost.  McCloud,  who  recognized 
the  merits  of  this  process,  has  now  formalized  this  method  for  Air 
Force  requirements. 

He  established  the  Air  Force  Requirements  Oversight  Council 
(AFROC),  chaired  by  the  Director  of  Operational  Requirements,  to 
look  at  every  Operational  Requirements  Document  sent  to  the  Air 
Staff  for  review.  The  AFROC  includes  representatives  from  the 
various  Headquarters,  Air  Force  offices,  including  Logistics; 
Acquisition;  Command,  Control,  and  Communications;  and  Test  and 
Evaluation,  as  well  as  from  organizations  outside  of  the  Pentagon, 
such  as  the  Air  Force  Operational  Test  and  Evaluation  Center,  Air 
Force  Materiel  Command,  and  the  major  command  sponsoring  the 
requirements.  The  stakeholders  of  the  requirements  process  serve 
as  members  of  the  AFROC.  In  effect,  the  new  AFROC  is  an  IPT  for 
Air  Force  requirements. 

One  of  the  side  benefits  of  the  new  review  process  is  that  more 
of  the  Air  Force  leadership  learns  about  the  various  needs  of  the 
service.  For  example,  in  the  past,  some  officers  never  learned  of  the 
need  for  space-based  sensors  or  improved  communication  satellites. 
Now  they  understand  the  requirements  and  can  better  weigh  the 
merits  of  future  investment  decisions.  Not  only  does  the  new  AFROC 
process  result  in  a  better  requirements  document,  one  that  identifies 
true  operational  needs,  but  it  also  results  in  the  “corporate  Air  Force” 
buying  into  what  the  Air  Force  needs  to  invest  funding  in. 

The  new  AFROC  process  captures  an  important  feature  of  the 
F-22  program — refining  requirements  to  address  the  basic  opera¬ 
tional  needs  in  a  way  that  gives  the  contractor  and  program  office 
the  flexibility  to  explore,  and  propose,  cost-effective  solutions.  Due  to 
the  AFROC,  this  requirements  concept  is  being  disseminated 
throughout  the  Air  Force  and  will  have  a  major  positive  impact  on 
future  Air  Force  programs. 
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A  New  Process  for  DOD  Acquisition 

Recent  actions  by  DOD  will  ensure  that  many  of  these  concepts 
receive  a  wider  audience.  On  May  10,  1995,  Secretary  of  Defense 
William  J.  Perry  announced  a  fundamental  change  in  the  way  the 
Department  will  acquire  weapon  systems.  In  a  memorandum,  he 
described  the  use  of  integrated  product  and  process  development 
(IPPD)  and  the  implementation  of  IPTs: 

The  IPPD  concept  has  been  successfully  used  by  the 
private  sector  and  by  the  Services  on  selected  programs  to 
reduce  product  cost  and  to  field  products  sooner. 

IPPD  is  a  management  technique  that  simultaneously 
integrates  all  essential  acquisition  activities  through  the 
use  of  multidisciplinary  teams  to  optimize  the  design,  man¬ 
ufacturing,  and  supportability  processes.  .  .  . 

IPTs  are  the  key  to  making  IPPD  work. 

IPTs  include  representatives  from  all  appropriate 
functional  disciplines  working  together  to  build  successful 
programs  and  enabling  decision  makers  to  make  the  right 
decisions  at  the  right  time.  IPTs  are  currently  being  used 
by  many  industry  and  government  program  managers.  ,  .  . 

I  am  directing  a  fundamental  change  in  the  way  the 
Department  acquires  goods  and  services.  The  concepts  of 
IPPD  and  IPTs  shall  he  applied  throughout  the  acquisition 
process  to  the  maximum  extent  practicable.  .  .  . 

Effective  immediately,  the  Department  shall:  Perform 
as  many  acquisitions  functions  as  possihle,  including  over¬ 
sight  and  review,  using  IPTs,  in  a  spirit  of  teamwork,  with 
participants  empowered  and  authorized  to  the  maximum 
extent  possible  to  make  commitments  for  the  organization 
or  functional  area  they  represent,  (italics  added  ) 

Perry^s  memo  included  a  list  of  10  IPPD  tenets  (see  Figure  10-1). 
Each  of  Perry’s  tenets  is  captured  in  some  aspect  of  the  F-22  princi¬ 
ples  (see  Figure  2-1),  and  several — early  and  continuous  life-cycle 
planning,  event-driven  scheduling,  multidisciplinary  teamwork, 
and  seamless  management  tools — are  exactly  the  same.  The  memo¬ 
randum  and  the  accompanying  IPPD  tenets  institutionalize  the 
concepts  proven  in  the  F-22  program. 

Reengineering  Acquisition  Oversight 

As  the  memorandum  states.  Secretary  Pony’s  goal  was  to  use  these 
concepts  at  all  levels  of  acquisition  in  as  many  areas  as  possible.  An 
innovative  application  of  IPTs  is  in  the  realm  of  oversight  and  review. 

A  chief  complaint  of  program  managers  had  been  that  the 
Pentagon  headquarters  staff  is  a  bureaucratic  roadblock  to  successful 
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development  of  a  weapon  system.  In  moving  forward  to  secure  a  mile¬ 
stone  decision,  which  allows  a  program  to  move  from  one  development 
phase  to  another  (such  as  from  demonstration/validation  to  engineer¬ 
ing  and  manufacturing  development),  program  directors  had  to  get 
the  approval  not  only  of  their  own  service  and  the  various  offices  in 
their  service  headquarters  (such  as  logistics,  test  and  evaluation, 
personnel,  and  operational  requirements),  but  also  of  the  OSD  and  its 
various  offices.  This  process,  known  as  Defense  Acquisition  Board 
review  and  approval,  could  be  time  consuming  and  painful.  Program 
directors  would  take  6  months  to  get  approval  and  might  have  to  brief 
more  than  50  individuals  or  groups.  The  directors  found  that  the 
headquarter  staff,  especially  the  OSD,  found  it  easy  to  veto  program 
approval  until  certain  conditions  were  met  (such  as  rewriting  test  and 
evaluation  master  plans  or  expanding  cost  and  operational  effective¬ 
ness  analyses).  The  headquarters  staff  seemed  to  be  the  real  enemy. 

The  Under  Secretary  of  Defense  for  Acquisition  and  Technology, 
Dr.  Paul  G.  Kaminski,  has  a  plan  to  improve  the  DAB  process: 
“Overarching  IPTs”  will  “structure  and  tailor  functionally  oriented 
IPTs  to  support  the  program  manager,  as  needed,  and  [aid]  in  the 
development  of  strategies  for  acquisition/contracts,  cost  estimates, 
evaluation  of  alternatives,  logistics  management,  etc.”  Each  major 
program  will  have  an  overarching  IPT  that  will  include  the  program 
manager,  the  program  executive  officer,  and  staff  principals  from  the 
OSD  and  the  appropriate  services.  The  appropriate  former  DAB 
Committee  Chair  (e.g.,  the  former  Strategic  Systems  Committee, 
Conventional  Systems  Committee)  will  lead  the  IPT.  Under 
Secretary  Kaminski  described  the  overarching  IPT  process. 

Figure  10-1.  The  Defense  Department’s  10  Tenets  of  Integrated 
Process  and  Product  Development 


1 .  Customer  focus 

2.  Concurrent  development  of  products  and  processes 

3.  Early  and  continuous  (ife-cycle  planning 

4.  Maximum  flexibility  for  optimization  and  use  of  contractor-unique 
approaches 

5.  Encouragement  of  robust  design  and  improved  process  capability 

6.  Event-driven  scheduling 

7.  Multidisciplinary  teamwork 

8.  Empowerment 

9.  Seamless  management  tools 

10.  Proactive  identification  and  management  of  risk 
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In  this  new  approach,  the  user,  the  program  memager, 
the  program  executive  officer,  the  service  component  staff, 
the  DOD  staff  and  related  decision  makers,  and  the  contrac¬ 
tor  involved  will  all  share  ownership  in  their  programs,  and 
they’ll  have  a  stake  in  making  the  program  successful.  .  .  . 

This  new  IPT  approach  is  different  in  that  it  involves 
early  Service  and  OSD  staff  involvement  at  the  start  of  the 
program — three  to  four  years  before  we’ve  gotten  to  this 
milestone  decision.  It  involves  teaming  with  the  program 
manager  and  the  program  executive  officer  to  develop  a 
quality  program  strategy  and  plan.  It  involves  a  joint  deter¬ 
mination  of  the  program  review  and  milestone  decision 
requirements;  and  a  joint  determination  of  the  functional 
IPT  requirements  and  the  documents — as  opposed  to  a  one- 
size-fits-all  approach:  that  is,  the  same  stamp-it-out 
approach  for  every  acquisition  program  that  we’re  taking. 

And  it  involves  early  and  joint  issue  identification  and  reso¬ 
lution  as  opposed  to  OSD  finding  fault  in  the  late  stages  of 
the  program. 

The  idea  behind  this  overarching  IPT  approach  is  to 
have  Service  and  OSD  staff  working  together  to  identify  and 
resolve  issues  early  in  the  program. 

I  think  the  result  of  this  process  will  be  to  provide  the 
best  possible  equipment  to  our  warfighters  in  a  more  effi¬ 
cient  and  cost  effective  manner. 

The  overarching  IPT  concept  allows  the  program  manager  to  tap 
into  a  knowledgeable  staff’s  accumulated  experiences  in  a  non¬ 
threatening  way.  The  OSD  and  service  staffs  imderstand  what  the 
constraints  of  the  program  are  early  on  and  learn  why  the  program 
manager,  or  user,  made  certain  key  decisions.  Overarching  IPTs 
should  improve  communication,  provide  a  forum  for  setting  realistic 
expectations,  ensure  ownership,  and  foster  a  win-win  relationship. 
In  short,  the  overarching  IPT  provides  the  forum  for  implementing 
the  principles  of  the  ATF/F-22  program. 

Can  Overarching  IPTs  Really  Work? 

Not  surprisingly,  as  soon  as  the  DOD  leadership  announced  the 
concept,  the  debate  immediately  started  as  to  whether  overarching 
IPTs  could  really  work.  At  an  initial  meeting  to  discuss  the  imple¬ 
mentation  of  IPTs,  many  program  managers  questioned  how  a  small 
program  office  could  work  with  a  larger,  overarching  IPT  containing 
members  from  many  parts  of  OSD  (who  might  outnumber  the  person¬ 
nel  in  the  program  office).  Others  questioned  the  role  of  members  from 
the  OSD  Comptroller’s  Office.  Would  their  interest  lie  in  success  of  the 
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program  or  in  finding  additional  ways  to  fund  budget  reductions?  Still 
others  wondered  how  members  of  the  OSD  staff  could  support  many 
IPTs  for  many  programs. 

The  characteristics  of  successful  IPTs,  described  in  chapter  7 
and  repeated  here  in  Figure  10-2,  form  the  basis  for  answers  to  these 
and  other  concerns.  The  most  important  characteristic  of  overarch¬ 
ing  IPTs  is  the  third,  that  they  are  empowered,  cross-functional 
teams.  The  members  of  the  overarching  IPTs  must  feel  they  have  a 
stake  in  the  successful  outcome  of  the  program.  If  they  can’t  add 
value  to  the  program  and  don’t  need  to  be  involved  in  it,  they  should 
not  be  part  of  the  overarching  IPT.  This  will  immediately  reduce  the 
potential  membership  of  the  overarching  IPTs, 

The  second  important  characteristic  of  overarching  IPTs  is  their 
commitment  to  the  customer  (typically  the  warfighter  in  the  field) 
and  their  focus  on  the  product — on  meeting  warfighter  needs 
through  successful  development  and  fielding  of  the  weapon.  Another 
important  point  is  that  the  overarching  IPT  needs  to  take  a  long¬ 
term  view  of  the  program.  Of  all  characteristics,  this  may  very  well 
be  the  hardest  to  instill  in  the  new  teams. 

Finally,  the  overarching  IPTs  need  to  imderstand  their  bound¬ 
aries — which  decisions  to  make,  which  to  refer  to  higher  levels  (such 
as  the  Defense  Acquisition  Executive),  and,  perhaps  more  important 
(at  least  from  a  program  director’s  perspective),  which  to  keep  at  the 
program  director  level.  The  IPTs  must  also  understand  how  their 
program  fits  into  the  overall  joint  warfighting  architecture.  With 
this  perspective  the  overarching  IPTs  can  help  guide  the  program 
toward  its  goal  of  fully  meeting  the  Joint  Task  Force  warfighting 
needs.  Because  of  this  final  characteristic,  I  believe  that  user  repre¬ 
sentation,  especially  from  the  Joint  Requirements  Oversight 
Council,  will  be  very  important  on  all  overarching  IPTs. 

As  in  any  endeavor,  a  major  ingredient  in  the  success  of  IPTs  will 
be  the  strength  of  the  leadership  at  the  OSD  and  service  levels  in 
ensuring  that  the  overarching  IPTs  actually  support  the  program 
director.  The  team  efforts  should  produce  value-added  results  that 
will  lead  to  a  successful  program. 

Will  overarching  IPTs  really  work?  With  good  leadership  in  and 
above  the  overarching  IPTs,  yes,  they  will.  It  will  .take  time,  and  no 
one  method  of  implementation  will  work  for  all  occasions.  But  the 
concepts  of  teamwork,  integration,  and  support  of  the  customer  will 
result  in  an  improved  acquisition  process  that  will  directly  benefit 
the  men  and  women  who  defend  the  interests  of  the  nation. 
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Figure  10-2.  Characteristics  of  Successful  IPTs 


Successful  IPTs 

1 .  Are  customer-focused 

2.  Are  product-oriented 

(Where  a  ''product"  can  be  an  item,  a  process  or  an  issue) 

3.  Are  empowered,  cross-functional  teams,  including 

•  All  stakeholders  Involved 

•  All  necessary  disciplines/organizations  (Including 
the  customer) 

4.  Take  a  long-term  view 

5.  Understand  the  limits  of  their  decisionmaking  authority 
(fenceposts) 


Summary 

The  F-22  program  has  served  as  an  outstanding,  successful 
example  for  the  acquisition  community  as  well  as  for  other  parts  of 
DOD.  The  tools  and  methods  crafted  in  this  fighter  program  have 
yielded  results  all  over  the  Department.  Overarching  IPTs  hold  the 
promise  of  greatly  improving  the  review  and  oversight  process  and, 
more  broadly,  shoxild  result  in  the  implementation  of  IPD  in  large 
and  small  programs  throughout  all  of  the  services.  Kaminski  was 
not  overstating  the  case  when  he  referred  to  Perry’s  May  10,  1995 
memorandum  as  ‘"fundamentally  changing  the  way  we  undertake 
our  processes  in  acquisition.” 
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PlKrto  courtesy  of  Lockneed-Martin. 

A  YF-22  rockete  skyward  in  an  aerial  loop.  When  the  advanced  tactical  fighter  was  first  conceived  some  people  doubted  that  an 
ATF  could  combine  stealth,  supercruise,  and  agility. 


''Integrated”  is  a  word  threatened  with  overuse  in  the  F-22pr0‘ 
gramme,  but  integration  is  what  sets  the  aircraft  apart  from 
previous  fighters.  The  F-22  will  be  developed  according  to  an 
integrated  master  plan,  which  sets  out  what  has  to  be  done, 
and  an  integrated  master  schedule,  which  says  when  it  has  to 
be  done.  Work  packages  will  be  handled  by  integrated  product 
teams  which  bring  together  all  the  disciplines  required. 

Flight  International 
(October  1,  1991,  Page  32) 


With  perhaps  teamwork  running  a  close  second,  the  word 
“integrated’’  certainly  is  the  most  prevalent  word  used  to 
describe  the  F-22  program.  And  rightly  so;  integrated 
teamwork  captures  the  essence  of  the  development  program.  This 
chapter  shows  how  the  F-22  program  approach  to  managing  risk, 
with  integrated  teamwork  at  its  core,  helps  control  the  complexity  of 
a  weapon  development  program. 

The  Challenge  of  Complexity 

Chapter  1  reviewed  some  of  the  historical  problems  with  the 
defense  acquisition  process:  staying  within  projected  cost,  meeting  the 
advertised  schedule,  and  performing  as  originally  required.  The  main 
reason  for  the  longstanding  theme  of  overruns,  delays,  and  worse- 
thsm-expected  performance  in  weapon  development  is  complexity. 

As  the  military-industrial  complex  developed  the  first  super¬ 
sonic  fighter,  radar-controlled  surface-to-air  missile,  computer- 
driven  destroyer  with  phased-array  radars,  and  hundreds  of  other 
weapon  systems,  it  ran  into  problems  both  never  encountered  before 
(mostly  technical)  and  familiar  (like  changes  in  requirements,  the 
use  of  incompatible  parts,  or  miscommunication).  The  reason  is  that 
weapon  systems  are  complex  and  their  development  is  complex. 

Complex  Weapon  Systems 

Weapon  systems  themselves  are  complex  because  they  are 
generally  made  up  of  a  vast  array  of  smaller,  highly  interrelated 
components.  For  example,  the  F-22  weapon  system  is  made  up  of  the 
aircraft  and  its  engine,  the  support  equipment,  and  the  information 
needed  to  repair  the  aircraft,  as  well  as  the  system  to  train  the  pilots 
and  maintenance  personnel.  The  aircraft  includes  a  vast  array  of 
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items,  including  the  avionics,  which  includes  a  radar,  flight  control 
systems,  hydraulically  controlled  actuators,  flight  control  surfaces, 
and  the  pilot  ejection  seat.  A  great  number  of  components  (and  sub¬ 
components)  must  all  work  together  correctly  to  allow  fighter  pilots 
to  successfully  complete  their  mission  of  air  superiority. 

Complex  Weapon  Development 

The  process  of  developing  a  weapon  is  also  complex.  To  develop 
a  weapon  system,  designers  are  faced  with  three  challenges.  First, 
they  must  understand  exactly  what  the  product  must  do  (in  the  case 
of  the  F-22,  to  detect  and  shoot  down  enemy  aircraft)  and  the  many 
constraints  on  the  system,  such  as  the  size  and  projected  cost  of  the 
system  or  the  equipment  the  system  must  work  with  (such  as  the 
Airborne  Warning  and  Control  System,  AWACS). 

The  second  challenge  is  to  create  an  initial  solution  to  meet 
the  requirements.  That  solution  generally  divides  up  the  tasks 
necessary  to  meet  the  top-level  requirements  and  assigns  these 
sub-requirements  to  various  pieces  of  hardware.  This  process  of  allo¬ 
cating  requirements  to  ever-lower  levels  of  indenture  continues  until 
the  designers  reach  the  most  basic  requirements.  For  example,  in 
some  parts  of  the  F-22  aircraft,  the  subcomponent  requirements 
nestle  down  14  levels  of  indent\ire. 

Finally,  the  designers  develop  the  specific  solution  to  meet  each 
subrequirement,  applying  technology  to  best  meet  each  subsystem 
need.  As  the  designers,  or  design  teams,  develop  their  particular  sub¬ 
system,  they  must  be  aware  of  how  it  relates  to  other  components,  and 
the  design  solutions  they  reach  must  allow  their  component  to  support 
and  work  with — ^that  is,  must  be  compatible  or  congruent  with — other 
components  of  the  overall  system  developed  by  other  teams.  For  exam¬ 
ple,  the  aircraft  radar  cannot  easily  pass  information  in  32-bit  words 
to  an  avionics  computer  designed  to  receive  16-bit  words. 

The  three-part  process — understand  top-level  requirements, 
allocate  requirements  to  lower  levels,  and  develop  solutions  to  lower- 
tier  requirements  with  the  correct  application  of  technology — 
sounds,  and  is,  difficult. 

Requirements.  Guaranteeing  that  the  designer  fully  under¬ 
stands  the  requirements  is  hard — not  just  because  of  communication 
challenges  but,  more  important,  because  the  designer  needs  to  make 
sure  the  group  setting  the  requirements  knows  what  it  really  wants. 
Pinning  all  of  the  requirements  down  is  frustrating  but  essential. 
The  designers  must  capture  the  user  expectations  or  the  user  will 
never  be  satisfied  with  the  final  product.  This  process  alone  drives 
the  complexity  of  weapon  development. 
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Technology.  Making  sure  that  new,  unproven  technology  used 
for  a  solution  works  the  way  the  designer  thinks  it  will  is  also  hard. 
Banking  on  a  technology  that  turns  out  not  to  meet  the  expected 
(and  planned  for)  characteristics  causes  great  turmoil  in  a  develop¬ 
ment  program.  Having  to  come  up  with  alternate  solutions  because 
of  problems  with  technology  affects  previously  made,  congruent  deci¬ 
sions.  The  consequences  of  using  unproven  technology  are  usually 
increased  cost,  a  delayed  schedule,  and,  frequently,  degraded  perfor¬ 
mance.  New,  unproven  technology  thus  adds  to  the  complexity  of 
development. 

Congruence.  Ensuring  that  teams  that  may  be  spread  across  the 
country  make  congruent  decisions  at  all  levels  is  hard  as  well.  There 
is  usually  more  than  one  right  answer  for  each  problem  or  question, 
but  overall  each  answer  must  relate  successfully  to  the  answers  to 
other  questions.  For  example,  the  radar  team  decision  must  match 
that  of  the  avionics  computer  team.  The  greater  the  number  of  and  the 
interrelationships  among  these  decisions,  the  greater  the  complexity 
of  the  development. 

Controlling  Complexity 

Logically,  the  way  to  successfully  develop  a  weapon  system  is 
for  the  program  manager  to  control  complexity  by  managing  the 
three  elements  of  the  process  above:  requirements,  technology,  and  a 
management  system  that  aids  in  making  congruent  decisions.  The 
program  manager  must  establish  a  process  to  determine  the  user's 
real  requirements,  eliminate  surprises  from  technology,  and  minimize 
the  risk  of  noncongruent  decisions. 

Risk  Management 

The  leadership  of  the  F-22  program  created  an  approach  to  risk 
management  that  has  successfully  controlled  the  risks  involved  in 
establishing  requirements,  managing  technology,  staying  on  schedule, 
meeting  performance  expectations,  staying  within  budgeted  cost,  and, 
most  important,  meeting  warfighter  expectations. 

As  outlined  in  chapter  3,  the  F-22  program  approach  to  risk 
management  includes  three  elements:  requirements  control,  technol¬ 
ogy  control,  and  the  use  of  a  proper  management  system  to  help 
ensure  congruent  decisions.  Figure  11-1  shows  this  structure. 

Requirements  Control 

The  ATF  program  iterated  requirements  development  process 
brings  the  warfighter  (the  user)  together  with  the  competing 
contractors  (see  chapter  5).  The  warfighter  sets  the  requirements, 
and  the  contractors  work  toward  meeting  the  requirements  under 
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the  constraint  to  use  only  proven,  or  at  least  attainable,  technology. 
Some  requirements  will  be  easily  met;  others  will  be  more  challeng¬ 
ing.  By  being  intimately  involved  in  this  early  phase  of  development, 
the  users  see  the  complexity  and  cost  of  meeting  their  stated 
requirements  and  can  reanalyze  them  to  see  if  there  is  another  way 
to  carry  out  the  mission  of  achieving  air  superiority  and  reducing  the 
cost  of  the  weapon  system.  Having  the  user  who  sets  the  require¬ 
ments  directly  involved  in  the  early  phases  of  a  program  helps 
control  complexity  by  ensuring  that  the  designers  know  exactly  the 
final  requirements  to  design  to. 


Figure  11-1.  Elements  of  Risk  Management 


Technology  Control 

In  the  ATF/F-22  program  the  contractors  could  not  propose  a 
solution  unless  they  had  demonstrated  that  the  desired  technology 
actually  worked  (see  chapter  6).  This  explains  why  each  team  con¬ 
ducted  numerous  demonstrations  and  tests,  including  a  flight  test 
program  using  full-scale  fl5dng  protot3rpes,  in  the  demonstration/ 
validation  phase  of  the  program.  By  essentially  prohibiting  unproven 
technology  from  the  program,  the  F-22  program  leadership  shielded 
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the  program  from  the  disturbance  and  complexity  caused  by  techno¬ 
logical  surprises. 

Management  System 

The  management  system  should  provide  a  disciplined  way  to  run 
the  program  with  the  ultimate  goal  of  ensuring  that  the  designers 
make  the  right,  congruent  decisions.  The  first  element  of  the  product- 
focused  management  system  consists  of  a  management  approach 
called  integrated  product  development  (IPD)  implemented  through 
the  use  of  multifunctional,  integrated  product  teams  (IPTs).  The 
second  element  of  the  management  system  is  an  integrated  tool  set 
that  supports  a  well-structured,  disciplined  program. 

Integrated  product  development.  All  of  the  experts  that  par¬ 
ticipate  in  developing  a  complex  weapon  system  apply  their  expertise 
to  resolve  myriad  issues  in  order  to  produce  a  product  that  meets  the 
customer’s  mission  requirements.  IPD  causes  these  experts  to  focus 
on  a  specific  portion  of  the  overall  weapon,  a  particular  product,  and 
mutually  create  a  solution  that  satisfies  the  issues  of  each  specialty 
(see  chapter  7  and  Figure  11-2).  IPD  requires  input  from  all  the  spe¬ 
cialties  (engineering,  logistics,  finance,  contracts,  etc.)  required  to 
design  a  specific  part  or  component,  lb  produce  a  product  that  will 
satisfy  its  customers,  the  team  of  experts  must  understand  both  their 
immediate  and  their  long-term  needs.  By  meeting  the  logistical,  engi¬ 
neering,  and  manufacturing  challenges  of  a  product  early  in  the 
design  phase,  the  team  greatly  reduces  later  problems  during  the 
product’s  lifecycle. 

A  successful  method  of  implementing  IPD  is  to  gather  the  experts 
associated  with  a  particular  product  into  a  dedicated,  product-focused 
team — an  IPT.  Most  IPTs  in  the  F-22  program  revolve  around  hard¬ 
ware  products,  such  as  the  landing  gear.  However,  the  concept  of  IPTs 
also  applies  to  nonhardware  products  such  as  processes  or  issues.  An 
important  lesson  learned  early  in  the  F-22  program  was  that  an  IPT 
with  the  responsibility  to  design  and  deliver  a  product  is  highly  moti- 
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Integrated  Product  Development 

• 

Is  customer-focused 

• 

Is  product-oriented 

• 

Uses  empowered,  cross-functional  teams 

• 

Takes  a  long-term  view 

vated  and  determined  to  create  the  best  design  for  the  product.  If 
team  constraints  (such  as  interfaces  with  other  systems,  maximum 
power  available,  allowed  costs)  are  not  clearly  defined,  the  IPT  will 
produce  an  excellent  product  that  may  not  work  correctly  in  the  over¬ 
all  system.  Therefore,  each  IPT  needs  to  understand  all  of  the  design 
constrednts  and  to  know  which  decisions  to  make  and  which  to  pass  to 
a  higher  level  of  IPT — that  is,  the  IPT  fenceposts.  Figure  11-3  shows 
the  characteristics  of  a  successful  IPT. 


Integrated  management  tools.  IPD  is  a  powerful  idea,  but  it 
needs  the  correct  tools  (see  chapter  8),  which  make  up  the  second 
element  of  the  management  system.  The  integrated  management 
framework,  developed  towards  the  end  of  the  ATF  demonstration/ 
validation  phase,  provides  the  features  needed  to  properly  control 
the  program  (see  Figure  11-4).  The  elements  of  the  framework 

(a)  plan  the  actions  and  commit  the  participants  of  the  program, 

(b)  monitor  and  track  the  actual  execution  of  the  program,  and 

(c)  provide  feedback  to  the  members  of  the  team.  Many  of  these  tools 
were  used  for  earlier  programs,  but  the  significant  difference  is 
that  the  F-22  program  coordinates  these  tools  by  focusing  them  on 
the  various  parts  of  the  weapon  system,  supporting  the  team  focus 
on  the  product.  The  F-22  program  leadership  also  mandated  a  single 
numbering  system,  making  it  easy  to  see  the  relationship  between 
program  elements;  for  example,  between  the  cost  and  schedule  of  a 
particular  item  under  development.  The  integration  of  the  manage¬ 
ment  tools  greatly  increases  the  usefulness  of  the  information 
available  to  the  teams  and  supports  a  disciplined  development 


Figure  11-3.  Characteristics  of  Successful  IPTs 


Successful  IPTs 

1 .  Are  customer-focused 

2.  Are  product-oriented 

(Where  a  “product"  can  be  an  item,  a  process  or  an  issue) 

3.  Are  empowered,  cross-functional  teams,  including 

•  All  stakeholders  involved 

•  All  necessary  discipiines/organizations  (including 
the  customer) 

4.  Take  a  long-term  view 

5.  Understand  the  limits  of  their  decisionmaking  authority 
(fenceposts) 
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process,  greatly  reducing  the  complexity  of  weapon  development. 
The  tools  are: 

Plan  and  commit:  As  stated,  the  process  starts  with  requirements 
(called  functional  system  specifications)  that  are  iteratively  devel¬ 
oped  with  the  warfighter  and  the  contractors  and  then  stated  in 
mission  terms  without  mandating  a  specific  solution.  The  contractor 
then  designs  the  program  along  product  lines.  The  single-numbering 
system  applied  here  is  then  used  throughout  other  portions  of  the 
program  (integrated  master  plans,  technical  performance  measures, 
cost/schedule  reports,  etc.),  resulting  in  a  structure  called  the  work 
breakdown  structure.  As  the  expert  at  designing,  building,  and  test¬ 
ing  a  weapon  system,  the  contractor  (with  review  by  the  Grovernment) 
prepares  the  statement  of  work  and,  finally,  determines  the  events 
that  need  to  occur  to  complete  the  statement  of  work  and  meet  the 
product  requirements.  The  contractor  prepares  this  detailed  plan  of 
the  events,  called  the  integrated  master  plan  (IMP),  before  work 

Figure  11-4.  The  F-22  Program  Integrated  Management  Framework 
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begins.  Ideally,  the  Government  program  manager  arranges  for  the 
IMP  to  be  created  during  the  competitive  phase  of  the  program  and 
then  use  it  as  part  of  source  selection.  Requiring  the  IMP  to  be  placed 
on  contract  results  in  contractor  execution  of  the  proposed  program. 

Track:  While  the  IMP  is  an  event-driven  plan,  which  shows  the 
sequence  the  program  shall  follow,  the  integrated  master  sche-dule 
(IMS)  provides  calendar-based  tasks  in  much  greater  detail  than  the 
IMP  does.  Because  program  schedules  can  often  change,  the  IMS  is 
not  put  on  contract  (otherwise  every  change  would  require  a  contract 
change).  The  IMS  serves  to  flesh  out  the  IMP  and  provides  the 
detailed  planning  necessary  to  coordinate  team  actions  and  track 
progress.  The  technical  performance  measures — the  product-focused 
metrics  used  to  track  technical  performance — show  the  performance 
goal  for  the  product,  the  projected  performance  and  the  current  per¬ 
formance  achieved  to  date,  and  a  history  of  achieved  and  projected 
performance.  The  cost  accounts  are  designed  to  report  along  work 
breakdown  structure  lines,  and,  because  the  work  breakdown  struc¬ 
ture  is  organized  along  product  lines,  the  cost  reports  are  directly 
comparable  to  the  IMS  and  the  technical  performance  measures.  By 
working  closely  with  the  contractors,  the  F-22  program  office 
receives  initial  “flash  reports”  on  cost/schedule  information  10  days 
after  the  contractors  close  their  monthly  accounting  books.  But  as 
detailed  as  the  IMS  is,  the  Government  and  the  contractor  must  still 
have  a  closure  plan,  an  agreement  stating  the  decisions  the  teams 
must  make  to  determine  when  an  item  can  be  declared  completed. 

Feedback:  The  F-22  engine  and  weapon  system  engineering  and 
manufacturing  development  contracts  are  cost-plus-award-fee  con¬ 
tracts.  This  means  that  a  Government  team  (program  office  and  other 
stakeholders)  reviews  the  contractor  performance  every  6  months. 
The  award  fee  official,  the  Air  Force  F-22  Program  Director,  then 
determines  the  amount  of  the  available  award  fee  the  contractor  will 
receive.  This  award  fee,  the  contractor  profit,  is  the  Program 
Director's  most  important  (really,  only)  tool  for  rewarding  success. 
Carefully  structuring  the  criteria  by  which  the  contractor  will  earn 
that  award  fee  allows  the  Government  and  the  contractors  to  simul¬ 
taneously  achieve  their  goals  (for  the  Government,  to  design,  build, 
and  field  a  reliable  jet;  for  the  contractor,  to  make  a  profit).  With  the 
integrated  tracking  and  reporting  tools,  docixmenting  the  performemce 
necessary  to  determine  the  every-6-month  award  fee  is  straightfor¬ 
ward  and  requires  very  little  overhead. 

The  F-22  management  system  relies  on  IPD  and  uses  integrated 
management  tools.  Both  elements  combine  to  form  a  disciplined 
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system  that  relies  on  the  core  principle  of  integrated  teamwork  with¬ 
in  and  among  product  teams  to  create  a  complex  weapon  system. 

Principles 

The  tools  and  techniques  developed  by  the  F-22  program  all  have 
a  common  underpinning  in  the  central  beliefs  or  principles  held  by  the 
ATF/F-22  leadership.  The  ATF/F-22  program  20  principles  of  acquisi¬ 
tion  (see  Figure  11-5),  discussed  in  chapter  2,  were  not  set  down  in 
stone  and  delivered  at  the  first  program  staff  meeting.  Instead,  they 
evolved  during  the  early  phases  of  the  program. 

The  principles  accurately  capture  the  most  important  goals 
that  the  F-22  program  leadership  wanted  the  Grovemment/contractor 
team  to  reach.  A  review  of  the  list  shows  that  there  is  nothing  amaz¬ 
ing  or  new  about  these  principles.  They  are  reasonable  and  straight¬ 
forward.  The  challenge  is  to  implement  them  so  that  they  support  one 
another  and  the  mission  of  the  organization. 

Lessons  Learned 

As  the  F-22  program  implemented  the  ideas  described  in  this 
book,  the  program  refined  its  approach  and  learned  about  the 

Figure  11 -5.  The  ATF/F-22  Program  Principles  of  Acquisition 


1.  Operate  with  integrity. 

2.  Work  as  a  team. 

3.  Use  logic  and  common  sense. 

4.  Involve  everyone. 

5.  Integrate  the  entire  system. 

6.  Ensure  ownership. 

7.  Use  a  disciplined  approach. 

8.  Understand  what  is  really  required. 

9.  Set  realistic  expectations  and  meet  them. 

10.  Provide  realistic  options. 

11.  Take  a  long-term  view. 

12.  Do  it  right  the  first  time. 

13.  Have  what  you  need  for  the  effort. 

14.  Ensure  everyone  knows  what  It  takes  to  meet  the  goal. 

15.  Use  an  event-based  schedule  with  defined  success  criteria. 

16.  Define  success  and  be  able  to  measure  it. 

17.  Reward  success. 

18.  Focus  on  a  win-win  relationship. 

19.  Guarantee  open  communications. 

20.  Achieve  success  with  a  positive  attitude  and  focus. 
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Figure  11-6.  Lessons  Learned  From  the  F-22  Program 

Program 

1 .  Full  teamwork  improves  program  performance. 

2.  Integrated  planning  reduces  start-up  and  supports  a  well- 
managed  program. 

3.  Integrated  management  tools  support  a  disciplined  process. 

4.  Tightly  managed  programs  suffer  greatly  from  unstable  funding. 

Integrated  Product  Development 

5.  IPD  requires  complete  senior-level  support. 

6.  IPD  requires  a  cultural  change. 

7.  Classic  functional  organizations  may  initially  resist  IPD. 

Integrated  Product  Teams 

8.  IPTs  naturally  accomplish  IPD. 

9.  IPT  members  are  motivated,  are  Involved,  and  have  ownership. 

10.  Every  IPT  requires  training. 

11.  IPTs  take  leadership  commitment  from  all  groups. 

12.  IPTs  should  have  experienced  and  empowered  members 

13.  The  I  in  IPT  can  easily  become  independent  instead  of  integrated. 

14.  IPTs  need  to  know  their  fenceposts. 

15.  Program  leadership  must  ensure  integration  across  the  IPTs  at 
every  level. 

16.  Communication  and  software  tools  help  integration. 

17.  IPT  managers  should  be  in  charge  of  personnel  and  budget 
resources. 

18.  Both  leadership  and  teams  must  set  and  track  IPT  objectives. 

19.  The  right  people  must  be  in  the  right  jobs  at  the  right  time. 


challenge  of  implementing  some  of  the  new  methods.  The  19  lessons 
learned  (see  chapter  9)  are  reiterated  in  Figixre  11-6.  Overall,  the  F- 
22  program  assessment  is  that  these  techniques  are  extremely  pow¬ 
erful  and  that  they  work.  Initial  start-up  and  acceptance  of  the  tech¬ 
niques  takes  senior  leadership  support.  I  believe  these  lessons 
learned  also  show  that  what  we  have  all  heard  from  the  earliest  time 
in  our  careers,  namely,  that  people  are  the  group’s  most  important 
resource,  is  absolutely  true.  People  make  it  happen. 

Final  Words 

The  F-22  program  application  of  principles — focused  on  a  disci¬ 
plined  process,  in  an  environment  of  integrity,  teamwork,  and 
logic — has  worked  well  to  meet  the  challenge  of  weapon  system  com¬ 
plexity.  The  value  of  these  basic  acquisition  truths  lies  in  the  fact 
that  they  work.  They  greatly  help  control  the  risks  of  a  complex 
weapon  development  effort. 
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This  book  has  described  how  to  manage  risk.  The  approach  devel¬ 
oped  by  the  F-22  program  works  well;  its  customer,  Air  Combat 
Command,  is  pleased;  and  the  principles  and  concepts  work  for  other 
programs  and  in  other  areas  as  well. 


Photo  coyrtesy  ol  Lodchood-Marto. 

Another  test  flight  of  the  YF-22  begins  as  a  pilot  taws  the  aircraft. 


Appendixes 


Appendixes 

A.  A  Brief  History  of  the  Advanced  Tactical 
Fighter 

B.  The  F-22  Test  Process 

C.  Prototypes 

D.  Source  Selection: 

Concepts  and  Lessons  Learned 


155 


Appendix  A 


A  Brief  History  of  the 
Advanced  Tactical 
Fighter 


•  The  Genesis  op  the  Aevanged  Tactical 
Fighter 

•  DEMONSTBATIOI^ALIDA'nON  FhASE 

•  The  FNOINSERING  AIH)  MANCFACTlntlNG 

Development  Phase 


156 


The  roads  we  traversed  were  dusty  and  crowded.  Vehicles 
moved  slowly,  bumper  to  bumper.  Fresh  out  of  West  Point, 
with  all  of  its  courses  in  conventional  procedures,  I  was 
offended  at  this  jamming  up  of  traffic.  It  wasn't  according  to 
the  book.  Leaning  over  Dad's  shoulder,  I  remarked,  Hou'd 
never  get  away  with  this  if  you  didn't  have  air  supremacy." 
I  received  an  impatient  snort,  ""Ifl  didn't  have  air  supremacy, 
I  wouldn't  be  here." 


Second  Lieutenant  John  Eisenhower, 
Normandy,  France,  June  1944 


In  more  than  40  years,  no  U.S.  ground  soldier  has  been  killed  by 
enemy  aircraft.  The  fundamental  lesson  to  be  learned  from  the 
history  of  aerial  warfare  is  the  importance  of  protecting  the  skies 
over  friendly  troops.  This  condition,  called  air  superiority  (or  in  its  full 
measure,  air  supremacy)  allows  ground  and  naval  forces  freedom  of 
movement,  prevents  disruption  of  their  activities,  and  protects  their 
lines  of  supply.  Air  superiority  also  provides  freedom  of  action  for 
friendly  air  forces.  The  U.S.  Air  Force  has  the  primary  responsibility 
for  this  demanding  task.  To  ensure  control  of  the  skies  for  the  Joint 
Forces  Commander,  the  Air  Force  in  the  late  1960s  developed  the 
F-15  air-superiority  fighter.  By  the  year  2000,  the  F-15  will  be  more 
than  25  years  old  and  will  be  outmatched  by  sophisticated  surface-to- 
air  missiles,  such  as  Russia's  SA-10  and  SA-12,  and  newer  foreign 
fighters,  such  as  France's  Rafale.  Td  maintain  its  air  superiority,  the 
Air  Force  is  developing  the  F-22  Advanced  Tactical  Fighter  (ATF). 

The  Genesis  of  the  Advanced  Tactical  Fighter 

One  can  trace  the  start  of  the  ATF  to  Headquarters,  Tactical  Air 
Command,  Langley  Air  Force  Base,  Virginia,  in  the  late  1970s.  The 
Air  Force  was  well  along  in  fielding  the  F-15  when  it  looked  ahead 
to  the  ever-improving  Soviet  fighters  and  surface-to-air  missiles. 
Given  the  long  lead  time  for  development,  the  Langley  planners 
knew  that  the  planning  and  design  of  a  new  generation  air- 
superiority  fighter  needed  to  start.  In  1981,  Colonel  John  Michael 
Loh,  later  the  Commander  of  Tactical  Air  Command,  wrote  the 
first  Tactical  Air  Force  Statement  of  Need  for  the  next-generation 
air-superiority  fighter. 

In  1981,  the  Defense  Resources  Board  approved  a  mission  element 
need  statement  that  directed  the  Air  Force  to  begin  development  of  a 
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Photo  courtesy  of  Lockheed-Martin, 

The  ATF  purpose  is  to  replace  the  aging,  but  highly  successful  F-15  in  the  air  superiority  role.  Here  an  F-15  (left)  flies  formatton 
with  the  YF-22  (center)  and  an  F-16  (right). 
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manned  air-to-air  fighter  with  funding  authorized  in  fiscal  year  1983, 
Air  Force  Systems  Command  commenced  work  on  a  host  of  technolo¬ 
gies  to  support  this  future  fighter:  new  approaches  to  avionics,  jet 
propulsion,  flight  controls,  and  airframe  materials.  The  Aeronautical 
Systems  Division,  the  Air  Force  Systems  Command  organization 
responsible  for  aircraft  development,  began  working  on  the  ATF  when 
it  issued  a  request  for  information  from  industry  on  possible  future 
fighter  concepts.  The  submitted  concepts  ranged  from  small,  agile 
fighters  to  large,  supersonic-cruising  YF- 12-like  aircraft.  The  Air 
Forceps  Scientific  Advisory  Board  Summer  Study  recommended  that 
the  service  pursue  an  F- 15-size,  supersonic-cruising,  low-observable 
aircraft,  and  the  Air  Force  organized  the  initial  system  program  office 
for  the  ATF  in  1983.  In  September  of  that  year,  the  program  office 
awarded  contracts  to  seven  aerospace  companies  for  the  concept 
development  investigation  phase:  Boeing,  General  Dynamics, 
Grumman,  Lockheed,  McDonnell  Douglas,  Northrop,  and  Rockwell 
International.  These  $1  million  firm-fixed-price  contracts  directed  the 
contractors  to  deUver  conceptual  designs  in  May  1984.  The  purpose  of 
this  phase  was  to  provide  the  information  necessary  to  select  the 
design  options  for  the  ATF  and  to  focus  the  needed  technologies  to  be 
explored  in  the  next  phase,  demonstration/validation.  The  proposed 
characteristics  of  the  ATF  included  supersonic  persistence  (later 
termed  supercruise — cruise  above  the  speed  of  sound  without  using 
afterburner),  increased  combat  radius  and  lethality,  improved  maneu¬ 
verability,  and  improved  supportability. 

In  parallel  to  the  airframe  concept  development,  the  Propulsion 
program  office  at  Aeronautical  Systems  Division  ran  the  Joint 
Advanced  Fighter  Engine  (JAFE)  program,  which  the  Air  Force  had 
started  in  recognition  that  it  took  longer  to  develop  jet  engines  than  to 
develop  the  airframes  they  would  power.  In  September  1983,  the 
Propulsion  program  office  awarded  firm-fixed-price  contracts  to 
General  Electric  and  Pratt  &  Whitney  to  build  two  demonstrator 
engines  incorporating  innovative  approaches  that  would  meet  the 
demands  of  decreased  engine  size,  higher  temperature  materials, 
increased  engine  thrust,  and  greatly  improved  overall  performance. 
The  Propulsion  program  office  changed  the  name  of  the  program  to 
the  ATF  Engine  (ATFE)  program  to  highlight  its  ultimate  objective. 
The  ATF  program  office  subsequently  assumed  control  of  the  ATFE 
program  to  give  the  ATF  Program  Director  complete  responsibility  for 
both  Government  efforts  (the  ATF  weapon  system  and  the  engine) 
necessary  to  the  success  of  the  fighter  development  program. 
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The  DemonstrationA^alidation  Phase 

In  May  1986,  in  the  midst  of  source  selection  for  the  demonstra¬ 
tion/validation  phase,  the  Air  Staff  updated  the  ATF's  program 
management  directive  to  include  the  development,  fabrication,  and 
testing  of  two  flying  prototype  aircraft,  called  Prototype  Air  Vehicles. 
In  October  1986,  the  Defense  Acquisition  Board  gave  the  ATF  program 
approval  to  enter  the  demonstration/validation  phase.  The  ATF  pro¬ 
gram  office  awarded  two  firm-fixed-price  airframe  contracts  to  two 
teams  of  contractors:  Lockheed  Aerospace  Systems  Corporation  (the 
first  prime  contractor),  located  in  Burbank,  California,  teamed  with 
General  Dynamics  Corporation  of  Fort  Worth,  Texas;  and  Boeing 
Military  Aircraft,  of  Seattle,  Washington.  Northrop  Aircraft  Division, 
of  Hawthorne,  California  (the  second  prime  contractor),  teamed  with 
McDonnell  Douglas  Corporation  of  St.  Loxiis,  Missouri.  The  Lockheed 
prototype  would  be  called  the  YF-22;  the  Northrop  plane,  the  YF-23. 
Each  contractor  team  would  build  two  of  their  prototype  aircraft,  one 
powered  by  the  General  Electric  prototype  engines  and  the  other,  by 
Pratt  &  Whitney  engines. 

The  ATF  program  office  then  modified  the  engine  contracts  with 
Greneral  Electric  Aircraft  Engine  Company  of  Cincinnati,  Ohio,  and 
Pratt  &  Whitney  Government  Engine  Business  of  West  Palm  Beach, 
Florida,  to  support  ATF  demonstration/validation.  This  modification 
required  delivery  of  six  ATF  prototype  engines  from  each  of  the  ATF 
engine  contractors,  four  to  power  two  prototypes  (the  YF-22  and  the 
YF-23  would  both  be  two-engine  aircraft)  and  two  to  remain  on  hand 
as  spares.  The  Pratt  &  Whitney  engine  would  be  known  as  the  F119; 
the  General  Electric  engine,  the  F120. 

From  1986  until  1990,  the  Air  Force  and  the  contractors  conduct¬ 
ed  an  intensive  requirements  trade-off  and  risk-reduction  process. 
The  prototypes  flew  in  1990,  providing  valuable  input  to  the  contrac¬ 
tors’  proposals  for  the  engineering  and  manufacturing  development 
phase  (previously  known  as  full-scale  development).  On  April  23, 
1991,  after  an  intensive  Air  Force  evaluation  of  both  airframe  and 
engine  proposals.  Secretary  of  the  Air  Force,  Dr.  Donald  Rice,  selected 
the  Lockheed  Aeronautical  Systems  Corporation  to  develop  the  F-22 
aircraft  and  Pratt  &  Whitney  Government  Engine  Business  to  devel¬ 
op  the  F119  engine.  (See  appendix  D  for  more  information  about  this 
source  selection.) 

The  Engineering  and  Manufacturing  Development  Phase 

On  August  2,  1991,  the  respective  contractors  and  members  of  the 
program  office  signed  the  cost-plus-award-fee  contracts  for  engineer- 
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ing  and  manufacturing  development.  As  befit  the  decision,  the  ATF 
program  office  became  the  F-22  program  office. 

The  following  week,  senior  personnel  from  the  F-22  program  office 
met  off-site  with  personnel  from  the  principal  companies  involved  in 
the  development  of  the  F-22  and  the  FI  19  and  key  members  of 
Tactical  Air  Command,  the  Air  Staff,  and  the  Office  of  the  Secretary  of 
Defense.  The  new  Commander  of  Tactical  Air  Command,  General 
Loh,  explained  his  expectations  for  the  new  F-22,  This  meeting  set 
the  tone  for  the  F-22  program.  The  emphasis  was  on  cooperation  and 
problem  solving.  The  members  of  the  team  explained  their  perspec¬ 
tives  and  their  concerns  about  the  program,  and  the  group  assigned 
action  items  to  resolve  these  concerns.  This  off-site  meeting  started 
the  engineering  and  manufacturing  program  off  on  the  right  track — 
with  an  attitude  of  open  communication,  trust,  and  cooperation. 

In  December  1991  the  Air  Vehicle  team  held  a  Require¬ 
ments/Design  Review  Update,  allocating  all  upper-level  requirements 
down  to  the  lowest  levels  to  allow  detailed  design  to  begin.  In  Jvme 
1992,  the  Air  Vehicle  team  held  the  Design  Review  Update,  marking  the 
completion  of  the  first  of  a  three-phase  design  cycle  and  the  transition 
from  parametric  weight  tracking  to  detailed  design  weight  tracking. 

In  July  1992,  the  F119  engine  completed  its  Critical  Design 
Review  and  its  Initial  Production  Readiness  Review,  which  completed 
39  separate  component  reviews.  The  team  validated  the  design  matu¬ 
rity  of  the  F119  and  approved  the  engine's  advancement  into  the  next 
development  phase,  engine  test. 

In  April  1993,  the  Air  Vehicle  team  conducted  its  Preliminary 
Design  Review.  Because  of  the  F-22  program's  commitment  to  event- 
based  schedules  (instead  of  calendar-based  schedules),  the  program 
had  delayed  this  review  from  its  originally  scheduled  date  in  January 
to  allow  the  team  to  work  out  critical  issues  such  as  design-to-cost, 
weight  reduction,  configuration  definition,  wind  tunnel  testing,  struc¬ 
tural  development  testing,  material  selection,  loads  definition,  and 
common  integrated  processor  maturation.  Completion  of  this  review 
signaled  the  end  of  the  second  phase  of  aircraft  design  and  the  begin¬ 
ning  of  the  third,  and  final,  phase  in  the  design  of  the  F-22  air  vehicle. 

In  March  1995,  the  Air  Vehicle  team  completed  its  design  and 
conducted  the  last  major  review,  the  Critical  Design  Review. 
Construction  of  the  first  flying  aircraft  based  on  the  final  design  is 
underway.  When  it  enters  the  inventory  around  2003,  the  F-22  will 
help  ensure  that  futme  joint  forces  commanders  will  enjoy  air  superi¬ 
ority  well  into  the  21“‘  century. 


The  F-22  Test  Process 


WniBJmyz  Modeling  Testing 
Ult^te  Test— Flight  Testing 
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If  we  see  something  out  of  the  ordinary  in  the  test  flight  we 
can  go  back  into  the  lab  and  recreate  the  event  to  try  to  isolate 
the  problem  .  .  .  rather  than  the  usual  fly,  fix,  fly. 

John  W.  Pieper, 
Lockheed  F-22  Chief  of  Test 


As  mentioned  in  chapter  2,  General  James  A.  Fain,  Jr.,  ATF 
Program  Director  from  1986  to  1992,  strongly  believed  that  all 
elements  of  the  ATF  development  program  needed  to  be  inte¬ 
grated.  As  a  former  test  pilot,  he  saw  the  need  to  tightly  integrate 
development  and  testing.  The  program’s  Grovemment  and  contractor 
test  managers  therefore  worked  closely  together  to  develop  the  F-22 
integrated  test  approach,  shown  in  Figure  B-1. 

The  test  process  starts  with  requirements,  in  the  upper  middle  of 
the  figure.  As  described  in  chapter  5,  the  operators  set  their  initial 
requirements,  which  led  to  fimctional  specifications,  which  in  turn  led 
to  the  contractor’s  preferred  system  design.  The  contractor  (and  in 
some  cases  the  Government)  then  "flew”  its  aircraft  design  through  a 
group  of  developmental  and  effectiveness  models,  which  showed  the 
performance  of  the  contractor  design.  Based  on  these  data,  the  con¬ 
tractor  revised  its  design  (shown  by  the  two-way  arrow  between 
Design  and  Models  in  the  figure).  When  the  contractor  identified 
requirements  that  could  not  be  met  in  a  balanced,  cost  effective  way, 
the  program  office,  the  operational  user,  and  the  contractor  together 
refined  and  updated  the  requirements.  This  occurred  thousands  of 
times  in  the  demonstration/validation  phase  and  has  continued  dur¬ 
ing  the  engineering  and  manufacturing  development  phase. 

After  fabricating  and  testing  the  initial  breadboard  components, 
the  contractors  used  the  data  to  validate  and  refine  their  model.  For 
example,  having  fabricated  the  first  very  high  frequency  radio  anten¬ 
na,  the  communications  subcontractor  tested  its  performance,  applied 
the  results  to  the  antenna  propagation  model,  and  adjusted  the  model 
to  produce  results  that  matched  the  laboratory  data.  The  process  con¬ 
tinued  for  the  next  level  of  indenture,  in  this  example,  the  communi¬ 
cations  subsystem,  which  might  include  the  antenna,  the  antenna 
interface  unit,  the  cable,  and  the  transmitter/receiver  brassboard 
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module.  The  next  level  of  model  included  transmitter  and  receiver 
efficiencies,  cable  line  loss,  and  other  important  aspects  of  the  subsys¬ 
tem’s  performance.  Ultimately  the  contractor  fabricated  a  full  avion¬ 
ics  system,  which  then  provided  data  used  to  refine  and  validate  the 
top-level  avionics  performance  models.  Through  this  iterative 
process,  new  top-level  performance  requirements  might  emerge,  caus¬ 
ing  the  contractor/program  office/operator  team  to  reexamine  the 
specification  and  perhaps  even  the  operational  requirements. 

The  greatest  benefit  of  this  evolutionary  testing  process,  from 
basic  component  modeling  and  testing  to  complex  system  modeling 
and  testing,  lies  in  risk  reduction  and  cost  control.  A  problem  caught 
during  component  testing  is  not  only  easier  to  find  than  it  would  be 
during  higher  level  testing  (since  the  component  is  less  complex  than 
a  higher  level  system)  but  is  also  easier  and  much  cheaper  to  fix.  Once 
the  component  passes  its  modeling  and  testing,  the  program  can  move 
up  to  the  next  level  of  subsystem. 

After  modeling  and  testing  components  through  system  hard¬ 
ware  comes  the  final  test — the  flight  test.  In  the  past,  when  pro¬ 
grams  did  not  perform  rigorous  system/subsystem  testing,  test 
pilots  would  catch  many  problems  during  the  flight  test.  In  a 
process  called  fly-fix-fly,  the  aircraft  would  stand  down  flight  opera¬ 
tions  while  the  contractor  and  program  office  found  a  solution  to  the 
problem.  After  the  contractor  fixed  the  test  plane,  flight  operations 
resumed.  Not  only  did  this  delay  the  flight  test  schedule  and  waste 
the  resources  of  the  flight  test  program,  but  it  was  very  expensive 
(relative  to  a  laboratory  environment)  to  find  these  problems  in  this 
phase  of  the  program.  The  F-22  program  has  found  that  lower  level 
testing  evolving  into  higher  level  testing  saves  time  (by  catching 
problems  early  on)  and  ultimately  costs  less  than  finding  routine 
development  problems  in  the  flight  test. 
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If  we  had  gone,  as  was  the  original  plan,  directly  into  a  full- 
scale  development  program  from  a  paper  demonstra¬ 
tion !  validation  program  [without  the  prototyping  effort]  we 
probably  would  have  a  pretty  big  schedule  and  cost  problem 
on  our  hands. 


Sherman  H,  Mullins, 
ATF  Program  Manager,  Lockheed, 

I  always  get  asked  about  prototypes.  Use  prototypes  only 
when  needed  to  reduce  risk. 


Greneral  James  A.  Fain,  Jr. 


Many  people  look  at  the  ATF  program  use  of  flying  protot3rpes 
and  draw  what  may  be  the  wrong  conclusions.  Major-scale 
prototypes  are  expensive,  and  their  inappropriate  use  can 
waste  critical  resources  that  could  be  better  applied  elsewhere. 
However,  protot3q>es  can  reduce  risk  and  save  millions  of  dollars.  This 
appendix  defines  prototypes  and  reviews  the  reasons  for  them,  the 
way  the  ATF  program  used  them,  and,  most  important,  the  reasons 
the  ATF  program  used  them. 

The  Tool  of  Prototypes 

Prototypes  are  a  tool  that  the  program  manager  can  use  to  reduce 
the  risk  in  the  program.  A  February  1993  Defense  Science  Board  Task 
Force  on  Aircraft  Assessment  defined  a  prototype  in  this  way: 

A  prototype  is  a  representative  model  used: 

1.  to  reduce  technical  risks  in  a  new  system  or 
subsystem 

2.  to  answer  design  questions  to  some  degree,  and 

3.  to  provide  necessary  confidence  before  moving 
to  the  next  phase  of  a  system  acquisition  with 
better  technical,  schedule,  and  cost  information 
and  estimates  for  the  system. 

In  discussing  prototypes  and  the  ATF  program,  many  people  tend  to 
think  exclusively  about  the  YF-22  and  YF-23  flying  prototypes. 
However,  the  ATF  program  used  many  types  of  prototypes,  for  example, 
the  ground  test  engines  in  each  engine  company  and  the  radar  cross- 
section  pole  models  (described  in  chapter  6).  Because  of  the  initial  high 
level  of  risk  in  avionics,  the  ATF^s  Avionics  Ground  Prototype  2Lnd  the 
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Photo  «MJi1esy  of  Lodcheed-Martin 

Prototypes  were  essential  in  the  validation/demonstration  phase  as  competing  firms  sought  to  establish  the  qualifications  of  their 
aircraft.  The  most  noticible  feature  of  the  Northrop  and  McDonnell  Douglas  YF-23  is  its  slanted  tall  stabllators. 


Photo  coyrtesy  of  Lockh^f-Martin. 

The  plantorm  view  of  the  Lockheed-General  Dynamics-Boeing  YF-22  prototype  displays  ite  twin  tail  stabilizers  and  wings. 
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avionics  Flying  Test  Bed  (integrated  brassboard  avionics  systems  that 
demonstrated  key  performance  capabilities)  were  perhaps  the  most 
important  prototypes  in  the  demonstration/validation  phase. 

Prototypes  can  be  more  than  just  hardware.  The  computational 
fluid  dynamics  model  of  an  aircraft,  the  “flying”  model,  can  be  viewed 
as  a  prototype.  The  engine  program  explored  the  most  innovative  pro¬ 
totype  of  the  ATF’s  demonstration/validation  phase — the  integrated 
master  plan  and  the  integrated  master  schedule  (see  chapter  8), 

The  1993  Defense  Science  Board,  in  an  analysis  of  the  F-22 
program,  said  that: 

Both  ground  and  flight  prototype  testing  in  the  DemATal 
phase  reduce  the  technical  risk  of  a  program,  thereby 
reducing  the  schedule  and  cost  risks  in  proceeding  to 
E&MD  [engineering  and  manufacturing  development] 

(and  production).  Prototyping  does  not  eliminate  techni¬ 
cal,  schedule,  and  cost  risk — that  is  why  there  is  an 
E&MD.  Prototypes  cost  money  and  take  time — sometimes 
they  are  justified  and  sometimes  not,  depending  on  the 
degree  of  technical  advance  sought  in  a  system  or  subsys¬ 
tem,  the  nature  of  the  technical  risks  and  the  costs  of 
risk  reduction  at  various  stages  of  an  E&MD  program. 

Clearly,  then,  one  should  use  prototypes  to  reduce  technical  as 
well  as  cost  and  schedule  risks. 

ATF  Prototype  Am  Vehicles 

As  a  result  of  the  Packard  Commission’s  Acquisition  Task  Force 
recommendation  to  increase  the  use  of  prototypes  (see  chapter  6), 
Headquarters  Air  Force  directed  the  ATF  program  office  to  include 
flying  prototypes  in  the  demonstration/validation  phase.  The  program 
office  fought  to  keep  the  requirements  for  the  prototypes  simple.  In 
fact,  the  only  Government  requirement  was  that  the  protot3q)e  actu¬ 
ally  take  off.  The  basic  concept  called  for  the  contractors  to  use  their 
protot3rpe  to  demonstrate  their  risk  areas.  The  contractors  would  then 
submit  the  data  from  their  flight  tests  as  part  of  their  engineering  and 
manufacturing  development  (EMD)  proposal  to  substantiate  their 
performance  projections  for  the  production  aircraft. 

In  an  interesting  sidelight,  a  key  member  of  the  Acquisition  Task 
Force  who  later  visited  the  program  office  to  review  the  program  was 
surprised  to  sec  that  the  program  included  fljdng  prototypes.  He 
believed  that  the  original  plan  of  testing  the  high-risk  avionics  and 
engines  was  sufficient  and  that  aeroperformance  risk  could  have  been 
adequately  addressed  by  modeling  and  simulation. 
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From  the  perspective  of  the  ATF's  Technical  Director,  Eric  E. 
Abell,  although  the  flying  prototypes  did  help  validate  the  computa¬ 
tional  fluid  dynamics  models  and  provide  data  unavailable  from  mod¬ 
eling,  such  as  testing  smoke  ingestion  by  the  engines  from  a  missile 
launch,  the  cost  of  the  prototype  air  vehicles  did  not  justify  their  use 
on  technical  risk-reduction  grounds.  However,  the  flying  prototypes 
did  serve  a  valuable  purpose  in  convincing  a  large  body  of  experts  that 
one  could  design  and  fly  a  high-performance  aircraft  designed  to  meet 
low-observable,  stealth  requirements.  An  unanticipated  benefit  was 
that  the  design  and  construction  of  the  prototype  air  vehicles  proved 
that  a  multicompany  team  could  design  and  produce  parts  of  an  air¬ 
craft  in  different  locations,  bring  them  together  on  time,  and  conduct 
a  highly  successful  flight  test  program.  Sherman  H,  Mullins, 
Lockheed^s  ATF  Program  Director,  summed  up  this  result: 

We  have  not  just  prototyped  an  airplane  or  avionics 
system.  We  have  prototyped  a  technical  integration  system 
that  clearly  demonstrates  that  you  can  make  these 
teaming  arrangements  work  in  a  technical  sense. 

Sealed  Envelope:  The  Importance  of  Modeling 

lb  build  and  fly  a  test  aircraft  during  the  demonstration/valida¬ 
tion  phase  required  the  contractors  to  finalize  their  aircraft’s  design 
well  before  all  of  the  operational  requirements  could  be  deter¬ 
mined.  From  the  outset,  the  ATF  program  leaders  recognized  that 
they  would  not  take  the  demonstration/validation  phase’s  Proto¬ 
type  Air  Vehicle  design  unaltered  into  the  EMD  phase.  However, 
beyond  using  the  original  fl3dng  prototypes  for  model  verifica¬ 
tion,  they  wanted  to  ensure  that  the  contractor  could  successfully 
predict  performance  and  thus  deliver  on  the  predicted  perform¬ 
ance  of  the  production  aircraft.  To  meet  this  objective,  the  program 
office  used  sealed-envelope  predictions.  According  to  General  Fain, 

Unless  your  requirements  are  firm,  if  you  use  prototypes, 
your  requirements  will  change.  Thus  you  must  imple¬ 
ment  the  sealed-envelope  approach.  You  must  establish 
the  contractor’s  ability  to  predict  and  verify  his  predic¬ 
tions  so  that  when  you  go  into  the  next  phase  (engineer¬ 
ing  and  manufacturing  in  our  case)  your  models  can 
predict  performance. 

Sealed  Envelope  Predictions 

Before  each  flight  test,  or  major  ground  test,  the  ATF  program 
office  required  the  contrator  team  to  submit  a  prediction  of  what  the 
test  data  would  show  at  the  conclusion  of  the  test.  The  program  office 
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wanted  specific  values,  such  as:  speed  at  altitude  for  sustained  super¬ 
sonic  cruise,  angle  of  attack,  roll  rate,  or  for  avionics  testing,  perhaps 
the  number  of  targets  acquired  and  processed  by  radar.  The  contrac¬ 
tor  submitted  this  data  in  a  sealed  envelope.  Following  the  test,  mem¬ 
bers  of  the  program  office  opened  the  envelope  and  examined  the  con¬ 
tractor’s  predictions. 

The  use  and  high  visibility  of  sealed-envelope  pretest  predictions 
helped  ensure  that  the  contractors  developed  and  validated  their  mod¬ 
els,  captiired  needed  test  data,  and  thoroughly  thought  through  the 
test  process  before  conducting  the  test. 

Summary 

The  ATF/F-22  program  has  shown  that  prototypes  can  be  a  very 
effective  tool  in  reducing  technical  as  well  as  cost  and  schedule  risk. 
However,  prototypes  can  be  much  more  than  just  flight-qualified  hard¬ 
ware:  Some  programs  may  use  hardware,  others  may  use  validated 
computerized  simulations.  The  ATF  program  also  showed  that  the  real 
value  of  flight  hardware  and  flight  testing  may  not  be  in  measuring 
aeroperformance  but  in  assessing  contractor  teaming  performance. 

Another  major  lesson  learned  is  that  future  program  managers 
should  ensure  that  the  contractor  can  successfully  predict  test  results. 
Confirming  a  model’s  accuracy  and  predictive  capability  can  greatly 
improve  the  chances  of  developing  a  final  product  that  will  meet  the 
operational  user’s  requirements. 
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Source  Selection:  Concepts 
and  Lessons  Learned 

*  GONtnEPTS 

•  1^  Mec«anics  OF  AIT  Soubce  Se^^ 


174 


Photo  courtesy  of  Lodtfieed-Martin* 


Raptor  01 ,  the  first  developmental  F-22,  takes  off  on  its  maiderf  flight. 


We  had  an  evaluation  to  make  of  two  competing  contractor 
proposals  and  two  teams.  Both  of  them  met  the  requirements; 
either  of  them  could  have  been  chosen  in  the  sense  that  they 
complied  with  what  the  Air  Force  had  asked  for,  so  we  had  to 
do  the  evaluation  based  on  looking  in  depth  into  the  way  they 
proposed  to  go  about  it,  into  their  management  plans,  into  the 
technical  risk  reduction  plans  that  they  presented.  We  went 
through  all  of  that  in  quite  a  lot  of  detail.  There  were  many 
factors  where  there  were  differences  between  the  two  teams, 
no  one  of  them  determinant  in  its  own  right,  but,  on  balance, 
we  concluded  that  the  Government  would  get  the  best  value 
by  proceeding  with  the  Lockheed  team  in  combination  with 
the  Pratt  &  Whitney  engine  manufacturer. 


Donald  B.  Rice, 
Secretary  of  the  Air  Force 


The  selection  of  the  contractor  to  develop  the  Advanced  Tactical 
Fighter  (ATF)  occurred  in  record  time  compared  with  earlier 
weapons  competitions.  Source  selection  evaluation  of^ici^llly 
began  for  the  engineering  and  manufacturing  development  (EMD) 
phase  when  the  Air  Force  released  its  request  for  proposals  (RFP)  on 
November  1,  1990.  The  four  competing  contractors  (Lockheed  and 
Northrop  for  the  ATF  weapon  system  and  Pratt  &  Whitney  and 
General  Electric  for  the  ATF  engine)  submitted  their  proposals  to  the 
ATF  program  office  on  January  2,  1991.  The  Government  completed 
its  evaluation  and  Secretary  of  the  Air  Force,  Donald  B.  Rice, 
annoimced  his  decisions  on  April  23, 1991,  Following  this  rapid  award 
of  one  of  the  largest  DOD  cost-plus-award-fee  contracts  ($11.2  billion 
for  weapon  system  development),  many  agencies  asked  what  lessons 
had  been  learned  from  this  highly  successful  source  selection.  The  fol¬ 
lowing  information  comes  from  two  unpublished  papers  written  by 
Mr.  James  D.  Schairbaum,  the  manager  of  ATF  source  selection;  and, 
Colonel  Wallace  T.  Bucher  and  myself. 

The  first  section  of  this  appendix  reviews  the  background  and  the 
concepts  of  the  ATFs  source  selection,  which  are  applicable  to  many 
other  acquisition  source  selections.  The  next  section,  The  Mechanics 
of  ATF  Source  Selection,  discusses  the  details  of  the  source  selection 
and  the  lessons  learned  that  support  the  concepts. 
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Concepts  Learned  From  Source  Selection 

The  ATF  commitment  to  integrated  teamwork  extended  through¬ 
out  the  preparation  for  the  EMD  phase.  In  April  1989  the  ATF’s 
Acquisition  Strategy  Working  Group  created  and  released  a  prelimi¬ 
nary  acquisition  package  that  laid  the  groundwork  for  what  would 
eventually  become  the  ATF  RFP.  The  contractors  scrutinized  this 
document  and  suggested  many  revisions  that  would  improve  how  the 
Government  asked  for  proposals.  The  working  group  released  the  first 
draft  of  the  RFP  in  August  1989  and  the  second  in  April  1990. 
Following  each  release,  the  contractors  met  individually  with  the 
Government  program  office  and  suggested  improvements,  which  the 
program  office  screened  to  ensure  that  they  added  value  to  the 
program  and  did  not  simply  assist  one  of  the  contractors.  Ultimately, 
the  draft  RFP  process  resulted  in  over  1,000  proposed  and  700  accept¬ 
ed  changes — greatly  improving  the  quality  of  the  Government’s  RFP 
and  giving  each  contractor  unparalleled  visibility  into  the  final  RFP. 

The  goal  of  the  Acquisition  Strategy  Working  Group  was  to  devel¬ 
op  and  explain  the  concept  of  integrated  master  plans  (IMPs)  and 
integrated  master  schedules  (IMSs).  The  working  group’s  strategy 
was  to  have  the  contractors  do  actual,  detailed  up-front  planning  by 
requiring  the  IMPs  to  be  part  of  the  contract  (submitted  as  part  of  the 

Figure  D-1,  ATF  Source  Selection  Concepts 


1.  Make  sure  the  leadership  is  thoroughly  familiar  with  all  parts  of  the 
request  for  proposal  (RFP) 

2.  Have  a  clear  and  agreed-to  understanding  of  the  requirements, 

3.  Understand  the  contractors’  program  and  ensure  that  it  has  the 
appropriate  level  of  risk. 

4.  Be  totally  prepared  for  source  selection  before  it  starts. 

5.  Guarantee  that  source  selection  participants  have  been  involved  in 
the  preparatory  activities  to  source  selection  and  are  stakeholders  in 
the  outcome  of  the  evaluation  and  the  ultimate  decision. 

6.  Take  the  necessary  steps  to  get  a  high-quality  proposal. 

7.  Ensure  that  all  high-level,  special-interest  groups  (which  are  backed 
by  law)  and  other  mandatory  exterior  groups  are  adequately  repre¬ 
sented  and  involved  prior  to  the  RFP  release. 

8.  Tailor  the  RFP  and  source  selection  process  to  meet  the  program’s 
particular  needs. 

9.  Have  all  Government  evaluators  conduct  a  “sealed-envelope”  evalu¬ 
ation  of  the  RFP,  including  the  contractor’s  predicted  response,  prior 
to  the  release  of  the  final  RFP. 
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proposal).  The  Government's  source  selection  team  thus  could  evalu¬ 
ate  the  contract,  which  would  be  in  force  once  the  contractor  was 
selected,  as  opposed  to  the  contractor’s  proposal,  which  merely  states 
the  contractor’s  intentions. 

The  ATF  program  office  learned  nine  key  concepts  as  a  result  of 
the  ATF  source  selection  (see  Figure  D-1). 

1.  Make  sure  the  leadership  is  thoroughly  familiar  with  all 
parts  of  the  RFP. 

One  final  authority  must  ensure  that  all  parties  have  included 
the  appropriate  requirements  in  the  RFP  and  that  all  elements  of 
the  RFP  have  the  correct,  common  focus.  This  individual,  perhaps 
supported  by  a  core  group,  prevents  a  stovepiped,  functional 
specialist  mentality  and  guarantees  that  all  appropriate  parties 
(including  the  user)  have  participated  in  the  RFP  process  (see 
Concept  7). 

2.  Have  a  clear  and  agreed-to  understanding  of  the  requirements. 

Both  the  system  program  office  (SPO)  and  the  contractors  must 
clearly  understand  the  user’s  functional  requirements.  The  compet¬ 
ing  contractors  must  also  thoroughly  and  correctly  understand  the 
SPO’s  additional  requirements  for  the  weapon  system  and  for  submit¬ 
ting  the  proposal. 

3.  Understand  the  contractors’  program  and  ensure  that  it 
has  the  appropriate  level  of  risk. 

The  key  to  evaluating  the  quality  of  a  contractor’s  approach  is  to 
understand  its  development  program  as  laid  out  in  its  proposal. 
Because  risk  is  the  major  issue  in  evaluating  a  proposed  program. 
Government  management  must  understand  the  contractor’s  proposed 
level  of  risk  and  evaluate  whether  it  is  appropriate  and  acceptable. 

4.  Be  totally  prepared  for  source  selection  before  it  starts. 

From  all  standpoints,  the  Government  team  must  be  fully  ready 
when  the  offerors’  proposals  arrive.  This  includes  knowing  who  is  on 
the  evaluation  team,  what  each  person  is  responsible  for,  and  how  the 
team  is  organized.  It  is  also  critical  to  fully  train  evaluation  person¬ 
nel  in  how  source  selection  will  operate,  how  they  will  do  their  job,  and 
how  they  will  report  and  document  their  portion  of  the  evaluation.  A 
detailed  schedule  should  include  everything  from  preparation  details 
to  interim  evaluation  submissions  and  contingencies  (such  as  a  period 
for  contractor  discussions). 

5.  Guarantee  that  source  selection  participants  have  been 
involved  in  the  preparatory  activities  to  source  selection  and 
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are  stakeholders  in  the  outcome  of  the  evaluation  and  the  ulti¬ 
mate  decision. 

In  the  best  of  all  possible  worlds,  the  people  who  make  up  the 
source  selection  team  will  be  same  people  who  wrote  the  draft  RFP, 
discussed  it  with  the  contractors,  and  will  have  to  live  with  the  chosen 
contractor.  This  involvement  really  supports  teamwork,  clear  com¬ 
munication,  and  an  understanding  of  the  other’s  view. 

6.  Take  the  necessary  steps  to  get  a  high-quality  proposal. 

Obviously  one  can’t  conduct  a  high-quality  source  selection  with¬ 
out  a  high-quality  proposal.  The  problem  is  identifying  the  way  to  get 
one.  The  contractors  must  know  that  they  must  submit  their  best  offer 
first  and  that  there  will  be  no  discussions. 

7.  Ensure  that  all  high-level  special-interest  groups  (which 
are  backed  by  law)  and  other  mandatory,  exterior  groups  are 
adequately  represented  and  involved  prior  to  the  RFP  release. 

In  general,  the  program  should  involve  every  stakeholder  before 
the  RFP  release — especially  those  who  have  the  ability  to  retard 
progress.  If  not  involved  up-front,  they  can  slow  down  source  selec¬ 
tion.  The  key  is  to  involve  them,  give  them  a  vote  in  the  contents  of 
the  final  RFP,  and  ensure  that  they  don’t  change  their  vote. 

8.  Tailor  the  RFP  and  source  selection  process  to  meet  the 
program’s  particular  needs. 

Some  in  the  source  selection  process  are  reluctant  to  modify  and 
adapt  all  portions  of  an  RFP  to  support  the  unique  needs  and  conditions 
of  the  program.  The  RFP,  especially  Section  L,  Instructions,  Conditions, 
and  Notices  to  Offerors  or  Quoters,  and  Section  M,  Evaluation  Factors 
for  Award,  can  and  should  be  tailored  to  the  program. 

9.  Have  all  Government  evaluators  conduct  a  ’’sealed-enve¬ 
lope”  evaluation  of  the  RFP,  including  the  contractor’s  pre¬ 
dicted  response,  prior  to  the  release  of  the  final  RFP. 

If  required  to  see  the  RFP  (or,  first,  the  draft  RFP)  from  the  offer¬ 
or’s  perspective  and  develop  an  estimate  of  what  the  contractor  will  pro¬ 
pose  to  do  or  is  capable  of  doing,  the  evaluators  will  prepare  a  better, 
more  realistic  RFP.  Forcing  the  Government  evaluators  to  answer  their 
own  words  shows  them  whether  what  they  are  asking  for  is  reasonable. 

The  nine  global  concepts  above,  which  were  the  keys  to  the  program 
office’s  success  in  source  selection  for  the  ATF’s  engineering  and  manu¬ 
facturing  development  phase,  are  applicable  to  most  other  programs. 
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The  Mechanics  of  ATF  Source  Selection 

The  goal  of  this  section  is  to  capture  the  important  details  of  the 
lessons  learned  from  the  1991  ATF  source  selection.  It  describes 
how  the  ATF  program  office  performed  the  tasks  of  source  selection 
(and  how  one  might  do  them  better).  Below  are  the  source  selection 
concepts  with  examples  from  the  ATF  source  selection.  The  specifics 
described  here  may  or  may  not  fit  other  programs,  depending 
upon  their  particular  situations. 

!•  Make  sure  the  leadership  is  thoroughly  familiar  with  all 
parts  of  the  RFP. 

The  Program  Director  served  as  the  final  authority  for  all  items 
going  into  the  RFP.  The  Program  Director  and  the  Deputy  Program 
Director,  Assistant  Program  Director,  Technical  Director,  Director  of 
Contracting,  and  Director  of  Program  Control  reviewed  the  draft  and 
final  RFPs  to  guarantee  that  the  items  requested  were  reasonable  and 
needed.  During  the  draft  RFP  phase,  they  also  reviewed  the  feedback 
from  the  offerors.  The  Program  Director  personally  made  sure  that 
the  data  items  and  the  required  specifications  were  reduced  to  the 
smallest  number  possible.  During  source  selection,  this  team  served 
as  the  key  leadership  of  the  Source  Selection  Evaluation  Board. 

2.  Have  a  clear  and  agreed-to  understanding  of  the  requirements. 

a.  Open  discussions  during  demonstration/validation. 

During  the  2  years  of  demonstration/validation  prior  to  the  RFP 
release,  the  program  office,  Tactical  Air  Command  (TAC),  Air  Training 
Command  (ATC),  the  Air  Logistic  Centers  (ALCs),  other  Government 
agencies,  and  the  contractor  teams  frequently  and  openly  discussed 
every  facet  of  the  RFP  and  the  EMD  phase  of  the  program.  It  was  only 
through  these  discussions  that  the  SPO  was  able  to  communicate  to 
the  contractors  and  other  Government  agencies  their  intentions, 
requirements,  and  procedures  for  conducting  source  selection. 

b.  Draft  RFPs.  The  ATF  program  office  started  the  draft  RFP 
process  2  years  prior  to  the  release  of  the  final  RFP  and  had  four  com¬ 
plete  draft  RFPs  (the  first  two  were  actually  called  Preliminary 
Acquisition  Packages  and  contained  sections  of  the  RFP)  that  they 
reviewed  with  the  offerors.  As  part  of  this  process,  the  SPO,  TAC,  the 
ALCs,  ATC,  and  the  National  Security  Agency  honed  the  require¬ 
ments  to  reflect  what  they  really  needed  and  phrase  them  adequately 
and  clearly.  As  a  result,  the  contractors  easily  saw  what  the 
Government  was  requesting.  A  key  part  of  this  process  was  the 
detailed  follow-up  to  each  draft  RFP.  Following  the  release  of  the  first 
two  draft  RFPs,  a  representative  of  each  Government  entity  and  a 
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contractor  representative  separately  discussed  the  RFP  and  reviewed 
the  contractor’s  suggestions  for  improvement.  Following  the  third 
and  fourth  (final)  draft  RFPs,  each  contractor’s  work  breakdown 
structure  group  formally  and  separately  submitted  final  comments  for 
improvement  to  the  SPO.  Also,  the  SPO  and  the  contractors  conduct¬ 
ed  monthly  in-process  reviews  to  formally  analyze  each  section  of  the 
RFP  During  this  process,  the  contractors  provided  over  1,000  docu¬ 
mented  comments  to  the  final  RFP  In  addition,  more  formal  acquisi¬ 
tion  strategy  working  groups  met  to  resolve  program-level  issues.  As 
a  result  of  the  four  iterations  of  draft  RFPs,  the  contractors  had  few 
or  no  exceptions  to  the  RFP,  the  Government  included  no  surprises  in 
the  final  RFP,  and  the  contractors  included  no  surprises  in  their  pro¬ 
posals.  Because  they  forced  the  Government  and  the  contractors  to 
communicate  clearly  and  understand  the  requirements,  the  draft 
RFPs  greatly  reduced  areas  requiring  discussion  and  assisted  in 
shrinking  the  time  necessary  to  conduct  the  source  selection. 

3.  Understand  the  contractors’  program  and  ensure  that  it 
has  the  appropriate  level  of  risk. 

a.  Proposal  **maps”  and  cross-references.  For  the  ATF  pro¬ 
gram,  the  key  to  understanding  a  complex,  multivolume  proposal  was 
to  prepare  a  map  of  the  proposal  and  an  in-depth  matrix  cross-refer¬ 
encing  the  requirements  to  sections  in  the  proposal.  A  standardized 
proposal  structure  (e.g.,  volume-numbering  schemes)  would  also  have 
aided  in  the  evaluation.  Requiring  such  review  aids  in  the  RFP  will 
help  ensure  thorough  proposal  evaluations,  which  will  lead  to  an 
understanding  of  the  contractor’s  proposed  program  and  proposed 
level  of  risk. 

b.  Evaluation  of  the  offeror’s  management  approach.  An 

extremely  helpful  procedure  was  to  evaluate  how  each  team  would 
manage  its  activities  to  develop  and  produce  the  ATF  weapon 
system.  This  evaluation  extended  to  the  contractor’s  management 
philosophy  as  well  as  its  organizational  relationships  with  team¬ 
mates  and  key  suppliers.  To  get  the  data  to  conduct  this  evaluation, 
the  program  office  tailored  RFP  Section  L,  Instructions,  Conditions, 
and  Notices  to  Offerors  or  Quoters. 

4.  Be  totaUy  prepared  for  source  selection  before  it  starts. 

a.  Factor  team  schedules.  Prior  to  the  receipt  of  proposals, 
the  factor  chiefs  had  to  state  the  seating  space  requirements  for  their 
team  (based  on  the  number  of  evaluators  on  the  factor  team)  and  their 
plan  for  reviewing  the  proposals.  Each  factor  schedule  had  to  support 
the  overall  area  schedule  of  a  quick-look  review  within  2  weeks  after 
proposal  receipt  and  a  detailed  review  by  each  team  within  1  month 
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following  receipt.  To  prepare  a  realistic  schedule,  most  factor  chiefs 
jointly  worked  out  the  required  events  with  their  factor  teams.  As  a 
result,  the  factor  evaluators  were  not  only  educated  in  what  would  be 
expected  of  them  during  source  selection  but  were  also  personally  part 
of  building  the  factor  schedule  and  thus  had  high  ownership  of  the 
process.  With  this  advance  planning  and  organization,  each  factor 
team  was  ready  to  go  on  the  first  day  of  source  selection  . 

b.  Training.  Formalizing  the  SPO  teams  that  had  been  work¬ 
ing  on  particular  areas  during  the  draft  RFP  process  for  the  previ¬ 
ous  2  years,  3  months  prior  to  the  receipt  of  proposals  the  SPO 
leadership  identified  each  member  of  each  factor  evaluation  team, 
each  item,  and  each  area  and  trained  these  individuals  on  the 
source  selection  process,  their  particular  job,  and  expectations  of 
them.  In  this  way,  all  evaluators  knew  what  factor  teams  they  were 
on,  who  their  factor  "boss”  was,  where  they  would  sit,  and  what  they 
would  do  during  source  selection. 

c.  Forms.  A  critical  part  of  the  planning  and  preparation  was 
creating  all  of  the  forms  needed  to  document  the  source  selection 
process  (e.g.,  Analysis  Worksheets,  Item  Summaries).  Before  source 
selection,  all  of  the  players  involved  (evaluators,  factor  chiefs,  team 
chiefs,  area  chiefs,  and  administrative  personnel)  should  design, 
review,  and  finalize  the  forms  so  that  they  cover  all  aspects  of  the 
source  selection  (initial  as  well  as  final  analysis)  The  purpose  and 
use  of  these  forms  should  be  taught  during  source  selection  training 
prior  to  the  start  of  source  selection. 

d.  Briefing  charts.  Careful  study  of  the  types  of  briefings 
required  during  source  selection  could  have  saved  a  great  deal  of 
preparation  and  translation  time  (transfer  of  information  from  one 
format  to  another).  The  approved  format  for  the  "master"  configura¬ 
tion  chart  should  be  finalized  prior  to  the  start  of  source  selection  so 
that  working-level  charts  for  factor  and  team  reviews  can  transfer 
more  easily  into  top-level  briefing  charts. 

e.  Notebooks.  Source  selections  are  heavily  bureaucratic  (by 
necessity  to  force  a  structured,  well-documented,  fair  selection 
process).  The  source  selection  leaders  (factor  chiefs  and  higher)  each 
were  given  a  notebook  that  contained  all  of  the  information  they  need¬ 
ed  to  perform  the  evaluation,  such  as  the  source  selection  regulations, 
the  source  selection  plan  (SSP),  the  source  selection  procedures  docu¬ 
ment  (SSPD),  a  cross-reference  matrix,  the  standards  and  sections  of 
the  RFP  that  applied  to  that  item  or  factor,  and  all  of  the  products  that 
team  members  would  complete  during  the  source  selection  process 
(e.g.,  analysis  worksheets  and  item  summaries  along  with  instruc¬ 
tions  on  how  to  fill  them  out).  Most  factor  chiefs  also  added  to  their 
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notebooks  their  factor  team’s  schedule  and  other  pertinent  informa¬ 
tion.  The  evaluation  team  found  these  notebooks  invaluable  in  orga¬ 
nizing  and  streamlining  how  they  worked. 

f*  Labeling  and  packaging  of  the  proposal.  Td  minimize  the 
administrative  time  required  to  receive  the  proposals  and  shelve  them 
so  that  evaluators  have  easy  access  to  them,  the  RFP  should  instruct 
the  offeror  how  to  label  and  package  the  proposal,  thereby  eliminating 
the  administrative  nightmare  of  unpacking  and  marking  the  propos¬ 
als  for  inventory  purposes.  To  ease  distribution,  the  RFP  should 
include  instructions  to  have  the  offeror  label  each  copy  of  each  volume 
(i.e.,  copy  1  of  25,  etc.),  and  to  have  the  offeror  provide  a  clear  and  logi¬ 
cal  invoice,  stating  the  specific  contents  of  each  box.  Additionally, 
offerors  should  be  instructed  to  submit  updated  copies  of  the  model 
contract  or  other  proposal  volumes,  if  required,  in  binders  of  a  differ¬ 
ent  color  than  the  binder  for  the  original  submittal  and  to  appropri¬ 
ately  annotate  the  changes  made  within  each  volume. 

g.  Provide  for  all  contingencies.  For  several  years  prior  to 
RFP  release,  the  clearly  stated  objective  of  the  ATF  source  selection 
was  to  conduct  the  selection  without  discussions.  The  SSP  and  the 
SSPD  followed  that  philosophy  and  did  not  include  provisions  or  pro¬ 
cedures  for  discussions.  When,  ultimately,  the  program  office  did 
require  discussions  with  the  contractor,  a  great  deal  of  effort  and  coor¬ 
dination  was  needed  because  the  procedures  had  not  previously  been 
thought  through.  The  lesson  is  to  prepare  for  the  worst-case  scenario 
when  creating  the  SSP  and  SSPD. 

h.  Evaluator  checklists.  All  evaluators  should  develop  check¬ 
lists  based  on  their  standards  and  RFP  instructions  and  have  them 
approved  by  their  factor  chief.  Factor  chiefs  can  provide  initial  gener¬ 
ic  checklists  to  each  evaluator  for  them  to  build  upon. 

5.  Guarantee  that  source  selection  participants  have  been 
involved  in  the  preparatory  activities  to  source  selection  and 
are  stakeholders  in  the  outcome  of  the  evaluation  and  the  ulti¬ 
mate  decision. 

Key  players — ^before,  during,  and  after.  In  the  ATF  source  selec¬ 
tion,  the  quality  of  the  proposal,  the  best  understood  contractor 
approach,  and  the  highest  quality  (most  complete)  evaluation  result¬ 
ed  when  one  person,  or  group  of  people,  wrote  the  RFP  section, 
reviewed  it  with  the  contractor  counterparts  (both  teams)  during 
the  multidraft  RFP  process,  created  the  standards  for  that  area’s 
proposal  evaluation,  and  chaired  (or  evaluated)  that  factor  during 
source  selection.  In  many  cases  these  individuals  were  to  work  on 
the  same  area  in  the  EMD  phase  and  were  thus  highly  motivated  to 
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thoroughly  understand  the  contractor's  program,  and  involving 
them  in  source  selection  greatly  helped  in  clear  communication  and 
understanding  of  requirements.  Conversely,  it  took  a  great  deal  of 
time  to  educate  evaluators  new  to  the  factor  being  evaluated  or, 
worse,  people  new  to  the  program  on  what  it  was  trying  to  accom¬ 
plish  and  how  the  program  was  trying  to  accomplish  it.  Their  eval¬ 
uations  were  not  as  complete  and  did  not  contain  nearly  the  same 
understanding  of  the  offerors’  proposed  approach  as  did  the  evalua¬ 
tions  written  by  those  who  had  been  involved  in  the  source  selection 
process  before,  during,  and  after  proposal  receipt. 

6.  Take  the  necessary  steps  to  get  a  high-quality  proposal. 

a.  No  discussions.  The  clearly  advertised  concept  in  the 
demonstration/validation  phase  was  to  conduct  a  no-discussions  EMD 
source  selection  to  convince  the  contractors  of  the  sincerity  of  the 
Government’s  requirements  and  to  engender  the  philosophy  of  ‘^do  it 
right  the  first  time.”  However,  the  Government  cannot  control  the 
requirement  for  discussion.  An  offeror  merely  needs  to  place  imrea- 
sonable  requirements  in,  for  example,  the  Government  Furnished 
Equipment  list,  and  discussions  would  be  required. 

b.  No  return  of  proposals.  As  a  way  of  reducing  the  repacking 
and  administrative  control  of  the  offeror’s  proposal  after  the  submittal, 
the  RFP  should  instruct  the  offeror  that  the  proposal  and  related 
documents  will  not  be  returned  unless  specifically  requested.  Although 
no  one  requested  them  back,  due  to  the  large  number  of  volumes  and 
copies,  it  could  have  been  an  administrative  and  cost  burden. 

c.  Restrictive  markings.  The  RFP  for  the  ATF  program 
should  have  instructed  the  offerer  not  to  put  any  restrictive  marking 
on  any  documents  in  the  proposal  that  are  to  be  included  in  the 
resulting  “model  contract.”  By  not  giving  guidance  to  the  offeror  in 
this  area,  the  RFP  created  a  potential  discussion  item.  Through  oral 
discussion  and  clarification  requests  (CRs),  the  offerors  had  to  be 
instructed  to  remove  the  m2ukings  so  contract  award  could  be 
made  using  the  offeror’s  submittals.  This  process  could  have  also 
been  an  administrative  burden. 

d.  Point  of  contact  for  government-furnished  property 
(GFP).  The  RFP  should  have  instructed  the  offerors  to  identify  the 
location  and  applicable  contract  and  to  provide  a  point  of  contact  for 
all  GFP.  The  information  necessary  to  track  requested  GFP  had  to  be 
obtained  through  CRs  and  caused  additional  oral  discussions.  A  more 
specific  RFP  might  have  avoided  this  problem  and  decreased  the  time 
spent  running  down  GFP. 

e.  Standards  that  reflect  the  RFP.  The  evaluators  should 
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prepare  their  section  of  the  RFP  and  then  prepare  the  evaluation 
standard.  This  helps  force  the  evaluators  to  ensure  that  if  they  want 
something  evaluated,  it  must  be  in  the  RFP. 

7.  Ensure  that  all  high-level,  special-interest  groups  (which 
are  backed  by  law)  and  other  mandatory,  exterior  groups  are 
adequately  represented  and  involved  prior  to  the  RFP  release. 

As  mentioned  above,  all  groups  (e.g.,  Small  Business  Office, 
Independent  Cost  Analysis,  Independent  Technical  Analysis)  that 
may  or  may  not  add  value  (but  can't  be  avoided)  should  be  involved 
early,  if  for  no  other  reason  than  to  ensure  that  they  do  not  retard 
the  program's  source  selection  schedule.  Also,  all  groups  and  reviews 
should  be  identified  well  ahead  of  RFP  release  so  that  the  necessary 
data  for  the  reviews  can  be  requested  or  generated  and  the  roles  and 
methods  can  be  defined. 

8.  Tailor  the  RFP  and  source  selection  process  to  meet  the 
program’s  particular  needs. 

The  “standard"  Section  M  did  not  specifically  address  the  contrac¬ 
tor's  organizational  structure  and  efficiency  within  the  specific  crite¬ 
ria.  Without  modifications  to  Section  M,  the  (Government  could  not 
have  fairly  evaluated  the  contractors'  integrated  product  team  imple¬ 
mentations.  The  program  office  modified  the  ATF  RFP  to  give  better 
insight  into  the  offeror's  organizational  relationship.  The  teaming 
relationship  is  likely  to  be  a  fact  of  life  for  the  foreseeable  future,  and 
organizational  efficiency  is  critical  to  the  success  of  the  program;  both 
deserve  special  emphasis  during  the  evaluation. 

9.  Have  all  Grovemment  evaluators  conduct  a  “sealed-enve¬ 
lope”  evaluation  of  the  RFP,  including  the  contractor’s 
predicted  response,  prior  to  the  release  of  the  final  RFP. 

SPO  estimates,  which  require  much  time  and  coordination  with¬ 
in  the  SPO,  and  expectations  provide  a  basis  for  reviewing  the  con¬ 
tents  of  the  proposals.  The  SPO  should  define  the  estimates  and 
expectations  prior  to  RFP  release  to  ensure  that  the  requirements  in 
the  RFP  are  sufficient  and  to  familiarize  source  selection  partici¬ 
pants  with  these  documents  prior  to  the  beginning  of  soxirce  selec¬ 
tion.  SPO  expectations  to  develop  include  software  development 
schedules,  expected  performance  values  (based  on  the  contractor's 
design  approach),  software  sizing  and  timing  estimates,  and  aircraft 
weight,  performance,  and  reliability.  All  SPO  estimates  (e.g.,  overall 
schedule,  software  sizing,  software  schedules)  and  expectations  (e.g., 
performance  and  schedule)  should  be  defined  prior  to  RFP  release. 
These  Government  estimates  will  be  very  helpful  in  independent  cost 
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assessments  as  well  as  provide  a  technical  check  against  what  is 
proposed  as  part  of  the  factor  evaluation. 

Other  Lessons  Learned 

Below  are  other  items  that  may  be  helpful  to  other  programs  as 
they  suffer  through  the  source  selection  process. 

1.  Think  through  the  discussions  process.  If  discussions 
turn  out  to  be  needed,  the  written  discussion  process  Clarification 
Requests  (CRs)  and  Modification  Requests  (MRs)  must  include  a  thor¬ 
ough  review  process  with  sufficient  feedback  to  originators. 

a.  CRs  and  MRs  required  numerous  reviews  before  their 
ultimate  release,  often  with  significant  changes  to  the  original  mean¬ 
ing.  In  the  CR/MR  review  process,  when  an  evaluator  originated  a  CR 
or  MR,  it  was  reviewed  by  the  evaluator’s  factor  chief,  item  captain, 
and  area  chairperson,  and  then  went  to  the  Source  Selection 
Evaluation  Board  for  approval  with  release  on  the  day  of  approval. 
The  CRs/MRs  could  be  sent  back  to  the  originator  for  rewrite  an3rtime 
during  this  process.  This  process  worked  extremely  well;  however,  in 
some  instances  CRs/MRs  required  numerous  rewrites,  in  part  because 
too  many  individuals  did  not  understand  the  CR/MR  generation 
process  prior  to  the  beginning  of  source  selection  and  wrote  insuffi¬ 
cient  and  incomplete  CRs/MRs.  CR/MR  training  could  reduce  the  need 
for  extensive  review.  Another  cause  of  CR/MR  rewrites  was  the  simi¬ 
larity  between  two  or  three  CRs/MRs,  sometimes  from  different  areas 
of  the  source  selection.  Upon  identification,  these  duplicative 
CRs/MRs  were  returned  to  the  originators  to  be  combined  into  a  sin¬ 
gle  document.  How  well  the  originators  communicated  with  each 
other  determined  how  much  of  the  original  information  remained  in 
the  resulting  single  CR/MR. 

b.  The  source  selection  team  did  not  provide  sufficient  feed¬ 
back  to  the  CR/MR  originators.  Some  were  lonaware  of  the  status  of 
CRs/MRs,  especially  when  they  involved  a  combined  Ttechnical/Program 
Structure  Area  input.  Tb  alleviate  this  problem,  a  computer-generated 
tracking  system  database  should  be  estabhshed  that  would  allow  every¬ 
one  in  source  selection  access  to  the  day-to-day  status  of  the  release  of 
CRs/MRs  and  the  receipt  of  the  offeror’s  response. 

2.  Organize  the  Past  Performance  Risk  Analysis  Group 
early.  The  Past  Performance  Risk  Analysis  Group  (PPRAG)’s  mission 
is  to  look  at  the  previous  performance  of  the  offerors.  Its  tools  include 
three  contractor  performance  assessment  reports  (databases  from 
Headquarters  Air  Force  Materiel  Command,  Aeronautical  Systems 
Center,  and  the  Defense  Plant  Representative  Office),  defense  acqui¬ 
sition  executive  summary  reports,  questionnaires,  and  interviews.  A 
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team  should  be  created  well  before  RFP  release  to  gather  the  infor¬ 
mation,  sort  and  cross-check  it,  and  start  the  past  performance  evalu¬ 
ation  prior  to  the  receipt  of  proposals. 

3.  Create  an  administrative  group.  The  ATF  program  office 
created  this  group  to  handle  all  of  the  paperwork  and  administrative 
support  (e.g.,  setting  up  local  computers,  unloading  and  shelving  pro¬ 
posals,  creating  forms,  tracking  CRs/MRs).  This  group  was  invaluable. 

Summary 

The  ATF  soiirce  selection  for  the  EMD  phase  was  remarkable  in 
its  efficiency  and  thoroughness.  It  was  also  remarkable  that,  with  so 
much  on  the  line,  neither  losing  bidder  protested  the  award.  It  is 
hoped  that  these  lessons  learned  will  help  future  programs  to  progress 
toward  successful  contractor  selection  as  well. 
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The  first  developmental  ¥-22,  Raptor  01,  during  its  first  flight. 
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